-- Logs begin at Wed 2022-08-24 17:01:17 UTC, end at Wed 2022-08-24 18:04:10 UTC. -- Aug 24 17:26:23.303376 dv-cinder-835384-24-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Aug 24 17:26:23.311543 dv-cinder-835384-24-iscsi devstack@keystone.service[81744]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Aug 24 17:26:23.316573 dv-cinder-835384-24-iscsi devstack@keystone.service[81744]: *** Starting uWSGI 2.0.18-debian (64bit) on [Wed Aug 24 17:26:23 2022] *** Aug 24 17:26:23.316573 dv-cinder-835384-24-iscsi devstack@keystone.service[81744]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 24 17:26:23.316573 dv-cinder-835384-24-iscsi devstack@keystone.service[81744]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 24 17:26:23.316573 dv-cinder-835384-24-iscsi devstack@keystone.service[81744]: nodename: dv-cinder-835384-24-iscsi Aug 24 17:26:23.316573 dv-cinder-835384-24-iscsi devstack@keystone.service[81744]: machine: x86_64 Aug 24 17:26:23.316573 dv-cinder-835384-24-iscsi devstack@keystone.service[81744]: clock source: unix Aug 24 17:26:23.316573 dv-cinder-835384-24-iscsi devstack@keystone.service[81744]: pcre jit disabled Aug 24 17:26:23.316573 dv-cinder-835384-24-iscsi devstack@keystone.service[81744]: detected number of CPU cores: 4 Aug 24 17:26:23.316573 dv-cinder-835384-24-iscsi devstack@keystone.service[81744]: current working directory: / Aug 24 17:26:23.316573 dv-cinder-835384-24-iscsi devstack@keystone.service[81744]: detected binary path: /usr/bin/uwsgi-core Aug 24 17:26:23.316573 dv-cinder-835384-24-iscsi devstack@keystone.service[81744]: your processes number limit is 47680 Aug 24 17:26:23.316573 dv-cinder-835384-24-iscsi devstack@keystone.service[81744]: your memory page size is 4096 bytes Aug 24 17:26:23.316573 dv-cinder-835384-24-iscsi devstack@keystone.service[81744]: detected max file descriptor number: 2048 Aug 24 17:26:23.316573 dv-cinder-835384-24-iscsi devstack@keystone.service[81744]: lock engine: pthread robust mutexes Aug 24 17:26:23.316573 dv-cinder-835384-24-iscsi devstack@keystone.service[81744]: thunder lock: enabled Aug 24 17:26:23.316573 dv-cinder-835384-24-iscsi devstack@keystone.service[81744]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Aug 24 17:26:23.316573 dv-cinder-835384-24-iscsi devstack@keystone.service[81744]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 24 17:26:23.354432 dv-cinder-835384-24-iscsi devstack@keystone.service[81744]: Python main interpreter initialized at 0x564e28dde9c0 Aug 24 17:26:23.354432 dv-cinder-835384-24-iscsi devstack@keystone.service[81744]: python threads support enabled Aug 24 17:26:23.354432 dv-cinder-835384-24-iscsi devstack@keystone.service[81744]: your server socket listen backlog is limited to 100 connections Aug 24 17:26:23.354432 dv-cinder-835384-24-iscsi devstack@keystone.service[81744]: your mercy for graceful operations on workers is 90 seconds Aug 24 17:26:23.354788 dv-cinder-835384-24-iscsi devstack@keystone.service[81744]: mapped 403077 bytes (393 KB) for 2 cores Aug 24 17:26:23.354867 dv-cinder-835384-24-iscsi devstack@keystone.service[81744]: *** Operational MODE: preforking *** Aug 24 17:26:23.355143 dv-cinder-835384-24-iscsi devstack@keystone.service[81744]: *** uWSGI is running in multiple interpreter mode *** Aug 24 17:26:23.355143 dv-cinder-835384-24-iscsi devstack@keystone.service[81744]: spawned uWSGI master process (pid: 81744) Aug 24 17:26:23.355348 dv-cinder-835384-24-iscsi systemd[1]: Started Devstack devstack@keystone.service. Aug 24 17:26:23.357269 dv-cinder-835384-24-iscsi devstack@keystone.service[81744]: spawned uWSGI worker 1 (pid: 81758, cores: 1) Aug 24 17:26:23.357269 dv-cinder-835384-24-iscsi devstack@keystone.service[81744]: spawned uWSGI worker 2 (pid: 81759, cores: 1) Aug 24 17:26:23.357431 dv-cinder-835384-24-iscsi devstack@keystone.service[81744]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 24 17:26:27.600445 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 24 17:26:27.600445 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: from cryptography.utils import int_from_bytes Aug 24 17:26:27.601521 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 24 17:26:27.601521 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: from cryptography.utils import int_from_bytes Aug 24 17:26:27.602831 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 24 17:26:27.602831 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: from cryptography.utils import int_from_bytes Aug 24 17:26:27.603276 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 24 17:26:27.603276 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: from cryptography.utils import int_from_bytes Aug 24 17:26:27.651659 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 24 17:26:27.651861 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 24 17:26:27.652056 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] command line args: [] {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 24 17:26:27.652331 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 24 17:26:27.652512 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] ================================================================================ {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 24 17:26:27.652621 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 24 17:26:27.652728 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 24 17:26:27.652817 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] admin_token = **** {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:26:27.652920 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] command line args: [] {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 24 17:26:27.653032 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] config_dir = [] {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:26:27.653188 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 24 17:26:27.653272 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:26:27.653355 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] ================================================================================ {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 24 17:26:27.653432 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] config_source = [] {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:26:27.653560 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] debug = True {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:26:27.653642 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] admin_token = **** {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:26:27.653758 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:26:27.654073 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] config_dir = [] {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:26:27.654073 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:26:27.654270 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:26:27.654367 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] config_source = [] {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:26:27.654466 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:26:27.654566 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] debug = True {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:26:27.654566 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:26:27.654782 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:26:27.654928 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:26:27.655031 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:26:27.655125 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:26:27.655234 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] list_limit = None {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:26:27.655337 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:26:27.655434 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] log_config_append = None {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:26:27.655434 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:26:27.655638 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:26:27.655735 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] log_dir = None {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:26:27.655837 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] list_limit = None {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:26:27.655933 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] log_file = None {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:26:27.656027 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] log_config_append = None {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:26:27.656127 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:26:27.656248 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:26:27.656248 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] log_dir = None {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:26:27.656471 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:26:27.656607 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] log_file = None {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:26:27.656716 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:26:27.656838 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:26:27.656960 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:26:27.657068 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:26:27.657188 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:26:27.657188 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:26:27.657405 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:26:27.657518 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:26:27.657638 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:26:27.657638 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:26:27.657863 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:26:27.657983 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:26:27.658098 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:26:27.658216 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:26:27.658338 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:26:27.658462 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:26:27.658574 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:26:27.658692 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:26:27.658806 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:26:27.658924 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:26:27.659038 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:26:27.659158 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:26:27.659272 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:26:27.659402 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:26:27.659522 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:26:27.659629 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:26:27.659629 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:26:27.659847 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.45/identity {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:26:27.659962 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:26:27.660075 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] publish_errors = False {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:26:27.660075 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:26:27.660303 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.45/identity {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:26:27.660409 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:26:27.660509 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] publish_errors = False {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:26:27.660645 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:26:27.660772 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:26:27.660886 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:26:27.660999 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:26:27.661111 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:26:27.661240 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:26:27.661362 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] standard_threads = False {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:26:27.661483 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:26:27.661597 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:26:27.661711 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:26:27.661823 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:26:27.661948 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] standard_threads = False {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:26:27.662063 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:26:27.662182 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:26:27.662296 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] use_journal = False {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:26:27.662410 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:26:27.662523 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] use_json = False {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:26:27.662685 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:26:27.662809 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] use_stderr = False {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:26:27.662924 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] use_journal = False {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:26:27.663037 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] use_syslog = False {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:26:27.663150 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] use_json = False {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:26:27.663262 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:26:27.663383 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] use_stderr = False {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:26:27.663383 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] use_syslog = False {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:26:27.663603 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.663718 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:26:27.663837 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.663963 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.664090 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.664205 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.664318 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.664438 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.664573 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.664690 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.664816 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.664937 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.665057 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.665171 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.665289 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.665410 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.665533 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.665653 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.665767 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.665882 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.665995 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.666107 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.666232 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.666347 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.666460 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.666579 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.666700 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.666819 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.666956 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.667072 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.667191 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.667312 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.667426 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.667545 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.667671 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.667787 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.667900 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.668013 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.668127 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.668239 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.668429 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.668545 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.668686 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.668807 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.668927 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.669056 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.669171 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.669283 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.669396 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.669509 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.669621 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.669741 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.669863 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.669977 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.670096 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.670210 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.670328 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.670451 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.670565 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.670677 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.670790 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.670902 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.671020 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.671144 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.671259 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.671378 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.671492 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.671604 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.671716 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.671837 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.671958 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.672072 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.672186 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.672298 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.672410 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.672531 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.672675 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.672788 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.672900 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.673023 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.673138 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.673260 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.673375 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.673494 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.673609 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.673721 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.673834 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.674024 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.674140 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.674253 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.674373 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.674494 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.674615 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.674737 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.674862 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.674977 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.675080 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.675182 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.675279 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.675383 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.675487 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.675583 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.675678 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.675779 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.675876 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.675979 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.676095 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.676216 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.676332 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.676446 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.676574 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.676705 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.676828 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.676943 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.677094 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.677210 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.677333 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.677450 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.677563 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.677677 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.677789 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.677902 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.678024 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.678140 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.678253 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.678375 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.678483 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.678602 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.678725 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.678851 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.678940 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.679064 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.679150 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.679222 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.679338 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.679414 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.679531 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.679647 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.679729 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.679846 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.679921 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.680043 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.680123 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.680196 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.680311 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.680393 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.680509 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.680613 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.680695 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.680815 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.680890 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.680961 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.681038 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.681111 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.681188 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.681261 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.681337 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.681408 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.681478 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.681548 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.681625 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.681698 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.681768 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.681885 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.681961 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.682032 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.682111 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.682184 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.682184 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.682315 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.682315 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.682441 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.682522 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.682594 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.682729 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.682809 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.682966 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.683043 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.683204 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.683283 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.683439 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.683516 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.683676 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.683753 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.683903 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.683979 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.684125 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.684202 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.684354 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.684442 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.684601 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.684698 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.684834 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.684918 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.685061 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.685137 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.685299 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.685384 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.685531 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.685611 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.685776 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.685854 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.686011 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.686096 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.686267 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.686346 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.686514 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.686591 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.686748 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] auth.external = None {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.686836 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.686979 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] auth.external = None {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.687064 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.687210 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.687297 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.687493 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.687580 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.687726 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.687811 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] auth.password = None {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.687956 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] auth.token = None {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.688040 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] auth.password = None {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.688182 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] auth.token = None {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.688267 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.688419 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.688511 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.688675 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.688756 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.688924 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.689002 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.689154 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.689243 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.689400 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.689487 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.689631 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.689709 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.689863 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.689949 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] credential.caching = True {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.690100 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] credential.caching = True {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.690179 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.690344 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.690429 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.690590 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.690678 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.690826 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.690904 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.691060 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.691138 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.691300 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.691386 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.691532 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.691610 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.691782 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.691861 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.692021 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.692108 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.692254 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.692332 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.692505 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.692748 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.692824 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.692911 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.693034 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.693113 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.693286 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.693364 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.693525 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.693611 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.693769 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] federation.caching = True {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.693847 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.694000 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] federation.caching = True {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.694078 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.694236 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.694323 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.694479 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.694558 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.694710 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.694787 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.694938 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.695023 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.695181 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.695262 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.695414 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.695494 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.695659 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.695776 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.695892 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.695970 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.696142 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.696226 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.696375 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.696455 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.696639 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] identity.caching = True {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.696720 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.696873 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] identity.caching = True {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.696951 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.697116 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.697197 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.697351 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.697429 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.698105 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.698195 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.698341 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.698417 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.698575 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.698655 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.698815 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.698898 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.699062 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.699141 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.699309 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.699389 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.699542 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.699620 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.699776 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.699860 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.700003 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.700080 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.700232 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.700311 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.700475 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.700611 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.700749 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.700827 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.700993 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.701080 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.701245 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.701325 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.701488 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.701567 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.701744 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.701821 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.701978 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.702099 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.702215 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.702329 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.702444 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.702558 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.702682 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.702798 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.702913 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.703026 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.703141 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.703257 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.703380 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.703498 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.703613 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.703729 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.703844 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.703959 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.704082 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.704198 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.704314 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.704428 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.704543 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.704682 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.704811 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.704929 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.705045 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.705160 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.705275 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.705398 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.705522 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.705638 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.705754 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.705875 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.705991 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.706115 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.706233 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.706348 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.706463 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.706578 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.706692 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.706817 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.706940 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.707057 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.707172 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.707287 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.707402 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.707525 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.707642 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.707642 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.707860 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.707975 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.708090 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.708213 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.708330 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.708454 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.708600 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.708719 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.708834 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.708963 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.709081 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.709197 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.709312 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.709428 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] ldap.user = None {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.709542 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] ldap.user = None {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.709666 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.709783 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.709783 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.710001 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.710117 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.710232 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.710360 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.710479 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.710595 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.710716 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.710839 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.710955 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.711080 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.711197 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.711312 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.711427 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.711542 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.711658 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.711781 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.711899 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.712015 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.712130 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.712245 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.712361 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.712488 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.712620 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.712737 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.712852 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.712967 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.713083 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.713808 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.713937 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.714053 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.714162 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.714276 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.714391 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.714516 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.714633 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.714748 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.714869 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.714986 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.715101 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.715230 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.715349 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.715349 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.715566 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.715681 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.715796 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.715921 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.716038 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.716154 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.716270 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.716385 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.716501 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.716653 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.716772 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.716888 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.717004 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.717120 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.717235 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.717361 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.717478 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.717594 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.717710 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.717826 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.717941 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.718066 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.718183 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.718299 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.718415 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.718531 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.718646 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.718770 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] resource.caching = True {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.718888 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] resource.caching = True {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.719004 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.719120 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.719235 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.719352 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.719477 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.719594 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.719710 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.719826 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.719941 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.720056 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.720180 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.720297 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.720414 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.720533 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.720715 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.720832 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.720960 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.721078 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.721195 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] role.caching = True {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.721310 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] role.caching = True {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.721426 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] role.driver = sql {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.721619 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] role.driver = sql {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.721739 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.721855 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.721970 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.722086 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.722202 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.722326 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.722444 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.722561 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.722677 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.722792 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.722908 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.723031 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.723149 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.723265 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.723380 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.723496 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.723613 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.723737 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.723854 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.723970 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.724085 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.724201 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.724316 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.724440 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.724577 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.724696 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.724813 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.724929 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.725044 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.725173 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.725291 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.725407 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.725522 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.725637 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.725754 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.726123 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.726245 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.726361 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.726476 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.726592 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.726717 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.726835 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.726957 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.727075 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.727191 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.727306 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.727430 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.727547 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.727663 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.727779 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.727895 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.728010 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.728134 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.728252 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.728368 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.728484 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.728613 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.728730 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.728860 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.728978 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.729095 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.729210 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.729325 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.729440 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.729564 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.729682 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.729798 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] token.caching = True {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.729914 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] token.caching = True {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.730030 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.730145 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.730268 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.730386 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.730505 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.730621 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.730736 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.730851 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.730977 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.730977 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.731196 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.731311 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.731426 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.731542 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.731665 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.731783 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.731904 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.732039 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.732156 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.732271 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.732395 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.732513 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.732646 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.732762 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.732877 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.732993 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.733121 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.733239 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.733354 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.733469 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.733584 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.733584 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.733814 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.733814 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] database.connection = **** {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.734042 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.734168 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.734285 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] database.connection = **** {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.734409 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.734535 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.734652 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.734767 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.734891 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.735008 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.735125 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.735240 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.735355 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.735470 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.735594 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.735712 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.735827 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.735942 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.736058 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.736174 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.736297 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.736415 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.736531 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.736665 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.736782 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.736897 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.737026 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.737144 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.737261 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.737377 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.737492 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.737607 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.737731 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.737849 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.737964 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.738079 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.738201 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.738319 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.738444 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81759) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 24 17:26:27.738562 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:26:27.738562 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81758) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 24 17:26:27.767055 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81759) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:26:27.767285 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81759) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:26:27.767513 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81759) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:26:27.767598 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81758) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:26:27.767768 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81758) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:26:27.767992 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81758) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:26:27.775913 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81759) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:26:27.776521 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81758) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:26:27.803725 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81759) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:26:27.803958 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81759) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:26:27.804147 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81758) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:26:27.804383 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81758) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:26:27.857987 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81759) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:26:27.858254 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81758) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:26:27.907216 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81759) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:26:27.907447 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81759) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:26:27.908304 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81758) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:26:27.908544 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81758) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:26:28.063290 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81759) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:26:28.063475 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81758) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:26:28.063559 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81759) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:26:28.063714 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81758) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:26:28.063823 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81759) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:26:28.063934 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81759) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:26:28.064017 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81758) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:26:28.064151 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81759) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:26:28.064236 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81758) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:26:28.064366 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81759) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:26:28.064459 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81758) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:26:28.064602 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81758) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:26:28.087252 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81759) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:26:28.087466 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81758) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:26:28.087557 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81759) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:26:28.087700 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81759) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:26:28.087782 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81758) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:26:28.087925 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81759) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:26:28.088008 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81758) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:26:28.088667 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81759) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:26:28.088771 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81758) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:26:28.088899 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81759) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:26:28.088983 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81758) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:26:28.089113 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81759) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:26:28.089196 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81758) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:26:28.089329 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81758) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:26:28.100229 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81759) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:26:28.100461 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81758) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:26:28.113433 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81759) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:26:28.113540 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81758) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:26:28.113618 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81759) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:26:28.113782 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81758) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:26:28.113864 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81759) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:26:28.113988 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81758) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:26:28.114068 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81759) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:26:28.114203 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81758) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:26:28.114296 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81759) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:26:28.114426 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81758) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:26:28.114509 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81759) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:26:28.114639 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81759) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:26:28.114722 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81758) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:26:28.114856 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81758) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:26:28.131529 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81759) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:26:28.131702 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81758) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:26:28.131787 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81759) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 24 17:26:28.131915 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81758) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 24 17:26:28.132063 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81759) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:26:28.132263 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81758) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:26:28.132347 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81759) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 24 17:26:28.132468 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81758) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 24 17:26:28.132570 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81759) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:26:28.132778 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81758) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:26:28.132872 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81759) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:26:28.133049 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81758) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:26:28.133309 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81759) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:26:28.133567 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81758) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:26:28.133649 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81759) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:26:28.133836 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81758) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:26:28.143425 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81759) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:26:28.143705 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81759) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:26:28.143788 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81758) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:26:28.143976 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81759) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:26:28.144060 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81758) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:26:28.144251 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81758) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:26:28.144334 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81759) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:26:28.144542 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81758) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:26:28.151676 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81759) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 17:26:28.152058 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81758) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 17:26:28.154556 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81759) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 17:26:28.154844 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81759) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:26:28.154983 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81758) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 17:26:28.155129 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81759) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:26:28.155273 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81758) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:26:28.155415 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81759) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:26:28.155562 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81758) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:26:28.155702 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81759) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:26:28.155851 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81758) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:26:28.155966 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81759) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:26:28.156142 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81758) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:26:28.156232 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81759) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:26:28.156412 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81758) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:26:28.156494 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81759) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:26:28.156715 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81758) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:26:28.156898 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81759) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:26:28.156992 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81758) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:26:28.157179 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81759) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:26:28.157375 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81758) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:26:28.157456 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81759) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:26:28.157659 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81758) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:26:28.157937 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81758) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:26:28.179680 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81759) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:26:28.180379 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81758) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:26:28.181166 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81759) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 17:26:28.181449 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81759) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:26:28.181880 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81758) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 17:26:28.182193 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81758) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:26:28.186103 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81759) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 17:26:28.186386 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81759) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:26:28.186669 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81759) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:26:28.186851 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81758) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 17:26:28.187140 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81758) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:26:28.187427 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81758) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:26:28.192953 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81759) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 17:26:28.193239 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81759) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:26:28.193744 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81758) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 17:26:28.194034 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81758) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:26:28.197274 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81759) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 17:26:28.198074 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81759) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:26:28.198362 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81759) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:26:28.198640 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81758) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 17:26:28.198724 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81759) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:26:28.198911 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81759) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:26:28.198992 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81758) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:26:28.199182 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81759) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:26:28.199268 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81758) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:26:28.199447 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81759) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:26:28.199540 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81758) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:26:28.199714 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81759) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:26:28.199795 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81758) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:26:28.200016 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81758) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:26:28.200290 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81758) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:26:28.200580 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81758) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:26:28.216623 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81759) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:26:28.216905 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81759) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:26:28.217534 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81758) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:26:28.217830 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81758) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:26:28.220760 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81759) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 17:26:28.221711 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81758) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 17:26:28.223947 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81759) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:26:28.224248 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81759) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:26:28.224943 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81758) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:26:28.225249 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81758) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:26:28.228703 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81759) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 17:26:28.230230 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81758) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 17:26:28.232327 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81759) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 17:26:28.233392 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81758) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 17:26:28.235555 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81759) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:26:28.235857 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81759) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:26:28.236142 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81759) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:26:28.236423 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81759) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:26:28.236655 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81758) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:26:28.236740 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81759) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:26:28.236966 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81758) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:26:28.237050 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81759) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:26:28.237260 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81758) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:26:28.237550 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81758) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:26:28.237843 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81758) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:26:28.238137 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81758) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:26:28.253477 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81759) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 17:26:28.253763 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81759) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:26:28.254030 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81759) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:26:28.254294 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81759) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:26:28.254374 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81758) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 17:26:28.254667 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81758) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:26:28.254952 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81758) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:26:28.255223 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81758) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:26:28.261110 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81759) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:26:28.262118 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81758) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:26:28.262621 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81759) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:26:28.262900 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81759) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:26:28.263647 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81758) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:26:28.263933 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81758) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:26:28.265507 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81759) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 17:26:28.265800 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81759) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:26:28.266092 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81759) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:26:28.266391 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81759) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:26:28.266577 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81758) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 17:26:28.266660 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81759) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:26:28.266872 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81758) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:26:28.267170 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81758) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:26:28.267466 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81758) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:26:28.267759 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81758) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:26:28.277924 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81759) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 17:26:28.278207 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81759) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:26:28.278491 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81759) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:26:28.278782 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81759) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:26:28.279047 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81759) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:26:28.279129 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81758) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 17:26:28.279322 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81759) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:26:28.279414 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81758) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:26:28.279605 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81759) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:26:28.279689 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81758) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:26:28.279909 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81758) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:26:28.280193 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81758) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:26:28.280482 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81758) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:26:28.280796 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81758) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:26:28.295289 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81759) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 17:26:28.296573 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81758) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 17:26:28.298342 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81759) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 17:26:28.299621 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81758) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 17:26:28.301223 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81759) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:26:28.302545 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81758) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:26:28.302656 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81759) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:26:28.303983 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81758) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:26:28.304123 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81759) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 17:26:28.304407 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81759) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:26:28.304724 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81759) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:26:28.305486 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81758) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 17:26:28.305773 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81758) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:26:28.306070 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81758) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:26:28.311663 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81759) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:26:28.313080 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81758) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:26:28.313166 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81759) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 17:26:28.314534 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81758) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 17:26:28.315984 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81759) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:26:28.316277 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81759) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:26:28.316636 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81759) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:26:28.316876 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81759) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:26:28.317524 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81758) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:26:28.317822 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81758) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:26:28.318111 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81758) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:26:28.318406 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81758) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:26:28.324759 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81759) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 17:26:28.325052 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81759) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:26:28.325334 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81759) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:26:28.326331 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81758) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 17:26:28.326626 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81758) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:26:28.326914 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81758) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:26:28.332142 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81759) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 17:26:28.332425 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81759) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:26:28.332736 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81759) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:26:28.333015 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81759) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:26:28.333296 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81759) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:26:28.333569 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81759) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:26:28.333844 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81759) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:26:28.333926 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81758) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 17:26:28.334131 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81758) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:26:28.334221 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81759) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:26:28.334408 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81759) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:26:28.334488 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81758) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:26:28.334685 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81758) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:26:28.334765 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81759) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:26:28.334954 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81759) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:26:28.335034 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81758) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:26:28.335224 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81759) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:26:28.335305 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81758) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:26:28.335486 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81759) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:26:28.335577 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81758) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:26:28.335753 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81759) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:26:28.335844 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81758) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:26:28.336053 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81758) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:26:28.336330 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81758) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:26:28.336639 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81758) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:26:28.336910 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81758) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:26:28.337188 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81758) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:26:28.337462 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81758) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:26:28.371517 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x564e28dde9c0 pid: 81759 (default app) Aug 24 17:26:28.372285 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x564e28dde9c0 pid: 81758 (default app) Aug 24 17:26:28.374976 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3028690-126b-4c86-8ea5-02ffe0827c3b None None] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:28.375271 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3028690-126b-4c86-8ea5-02ffe0827c3b None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:28.375548 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3028690-126b-4c86-8ea5-02ffe0827c3b None None] PATH_INFO: `/v3` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:28.379737 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1/1] 10.222.0.45 () {50 vars in 791 bytes} [Wed Aug 24 17:26:28 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:26:34.936270 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c672ab86-4bc8-4b53-8344-f295a9b36ecd None None] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:34.936798 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c672ab86-4bc8-4b53-8344-f295a9b36ecd None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:34.936878 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c672ab86-4bc8-4b53-8344-f295a9b36ecd None None] PATH_INFO: `/` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:34.941116 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1/2] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:26:34 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 17:26:34.947498 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece86223-bece-4987-a3b0-89209c54b92d None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:34.948379 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece86223-bece-4987-a3b0-89209c54b92d None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:34.948533 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece86223-bece-4987-a3b0-89209c54b92d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:35.018149 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: INFO dbcounter [None req-ece86223-bece-4987-a3b0-89209c54b92d None None] Registered counter for database keystone Aug 24 17:26:35.037243 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG oslo_db.sqlalchemy.engines [None req-ece86223-bece-4987-a3b0-89209c54b92d None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81759) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 24 17:26:35.043437 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writer thread running {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 24 17:26:35.219637 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG passlib.handlers.bcrypt [None req-ece86223-bece-4987-a3b0-89209c54b92d None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=81759) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 24 17:26:35.220055 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG passlib.handlers.bcrypt [None req-ece86223-bece-4987-a3b0-89209c54b92d None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81759) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 24 17:26:35.274244 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-ece86223-bece-4987-a3b0-89209c54b92d None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:26:35.275894 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-ece86223-bece-4987-a3b0-89209c54b92d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:26:35.403776 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 2/3] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:26:34 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 457 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 24 17:26:35.880630 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c672ab86-4bc8-4b53-8344-f295a9b36ecd None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:35.948880 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: INFO dbcounter [None req-c672ab86-4bc8-4b53-8344-f295a9b36ecd None None] Registered counter for database keystone Aug 24 17:26:35.970040 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG oslo_db.sqlalchemy.engines [None req-c672ab86-4bc8-4b53-8344-f295a9b36ecd None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81758) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 24 17:26:35.976138 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writer thread running {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 24 17:26:36.094102 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c672ab86-4bc8-4b53-8344-f295a9b36ecd None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:36.098356 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1bd8b5-ba76-4df2-97e9-045bdac6d1a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:36.098985 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1bd8b5-ba76-4df2-97e9-045bdac6d1a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:36.099273 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1bd8b5-ba76-4df2-97e9-045bdac6d1a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:36.099555 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1bd8b5-ba76-4df2-97e9-045bdac6d1a5 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:36.108321 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1bd8b5-ba76-4df2-97e9-045bdac6d1a5 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:36.159589 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: WARNING keystone.common.rbac_enforcer.enforcer [None req-fb1bd8b5-ba76-4df2-97e9-045bdac6d1a5 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 24 17:26:36.159940 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1bd8b5-ba76-4df2-97e9-045bdac6d1a5 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:36.170020 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 2/4] 10.222.0.45 () {58 vars in 1324 bytes} [Wed Aug 24 17:26:35 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 24 17:26:36.175941 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece86223-bece-4987-a3b0-89209c54b92d None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:36.190184 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece86223-bece-4987-a3b0-89209c54b92d None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:36.194580 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c61f386-7583-4e2b-8a30-4448f096bf39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:36.195695 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c61f386-7583-4e2b-8a30-4448f096bf39 admin admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:36.195979 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c61f386-7583-4e2b-8a30-4448f096bf39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:36.196254 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c61f386-7583-4e2b-8a30-4448f096bf39 admin admin] PATH_INFO: `/v3/projects/1e847003e2ef47df81d36ac4caad47f0` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:36.197852 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c61f386-7583-4e2b-8a30-4448f096bf39 admin admin] RBAC: Authorizing `identity:get_project(project_id=1e847003e2ef47df81d36ac4caad47f0)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:36.250968 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: WARNING keystone.common.rbac_enforcer.enforcer [None req-0c61f386-7583-4e2b-8a30-4448f096bf39 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 24 17:26:36.251315 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c61f386-7583-4e2b-8a30-4448f096bf39 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:36.255821 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 3/5] 10.222.0.45 () {56 vars in 1207 bytes} [Wed Aug 24 17:26:36 2022] GET /identity/v3/projects/1e847003e2ef47df81d36ac4caad47f0 => generated 344 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:26:38.162878 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fee8907-b666-4f05-9658-04ed0d23d422 None None] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:38.163426 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fee8907-b666-4f05-9658-04ed0d23d422 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:38.163426 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fee8907-b666-4f05-9658-04ed0d23d422 None None] PATH_INFO: `/` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:38.167584 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 3/6] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:26:38 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:26:38.173074 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec9b889-2bd2-47ce-acd2-f57fba46ec9e None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:38.173513 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec9b889-2bd2-47ce-acd2-f57fba46ec9e None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:38.173606 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec9b889-2bd2-47ce-acd2-f57fba46ec9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:38.193068 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-1ec9b889-2bd2-47ce-acd2-f57fba46ec9e None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:26:38.194569 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-1ec9b889-2bd2-47ce-acd2-f57fba46ec9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:26:38.202717 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 4/7] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:26:38 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 24 17:26:38.469998 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fee8907-b666-4f05-9658-04ed0d23d422 None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:38.482803 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fee8907-b666-4f05-9658-04ed0d23d422 None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:38.487040 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8397fc6f-bfee-4942-8f41-f736514afc83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:38.487642 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8397fc6f-bfee-4942-8f41-f736514afc83 admin admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:38.487925 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8397fc6f-bfee-4942-8f41-f736514afc83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:38.488204 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8397fc6f-bfee-4942-8f41-f736514afc83 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:38.495756 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8397fc6f-bfee-4942-8f41-f736514afc83 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:38.497577 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8397fc6f-bfee-4942-8f41-f736514afc83 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:38.506205 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 4/8] 10.222.0.45 () {58 vars in 1324 bytes} [Wed Aug 24 17:26:38 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 17:26:38.511373 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec9b889-2bd2-47ce-acd2-f57fba46ec9e None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:38.520309 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec9b889-2bd2-47ce-acd2-f57fba46ec9e None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:38.524575 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f06e325-bffa-4452-8637-5ca5f28d6a8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:38.525202 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f06e325-bffa-4452-8637-5ca5f28d6a8f admin admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:38.525482 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f06e325-bffa-4452-8637-5ca5f28d6a8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:38.526394 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f06e325-bffa-4452-8637-5ca5f28d6a8f admin admin] PATH_INFO: `/v3/users/cc2669e1aa494dbcb7164d2e5e46378c` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:38.527703 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f06e325-bffa-4452-8637-5ca5f28d6a8f admin admin] RBAC: Authorizing `identity:get_user(user_id=cc2669e1aa494dbcb7164d2e5e46378c)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:38.529534 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f06e325-bffa-4452-8637-5ca5f28d6a8f admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:38.531189 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 5/9] 10.222.0.45 () {56 vars in 1198 bytes} [Wed Aug 24 17:26:38 2022] GET /identity/v3/users/cc2669e1aa494dbcb7164d2e5e46378c => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:26:40.467293 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9efddde-200d-4884-8ba3-5553b10b7b2c None None] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:40.467831 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9efddde-200d-4884-8ba3-5553b10b7b2c None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:40.467831 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9efddde-200d-4884-8ba3-5553b10b7b2c None None] PATH_INFO: `/` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:40.469203 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 5/10] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:26:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:26:40.474437 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f875a457-a340-42d3-a3b2-0ebfdc90ee0f None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:40.474853 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f875a457-a340-42d3-a3b2-0ebfdc90ee0f None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:40.475000 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f875a457-a340-42d3-a3b2-0ebfdc90ee0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:40.492755 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-f875a457-a340-42d3-a3b2-0ebfdc90ee0f None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:26:40.493928 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-f875a457-a340-42d3-a3b2-0ebfdc90ee0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:26:40.515073 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aebcd59-8c67-48de-b9c4-247c3becf917 None None] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:40.515359 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aebcd59-8c67-48de-b9c4-247c3becf917 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:40.515634 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aebcd59-8c67-48de-b9c4-247c3becf917 None None] PATH_INFO: `/` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:40.516987 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 6/11] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:26:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 17:26:40.521921 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8501560-b4a0-4451-b059-f747861f30a0 None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:40.522197 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8501560-b4a0-4451-b059-f747861f30a0 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:40.522471 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8501560-b4a0-4451-b059-f747861f30a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:40.563720 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG passlib.handlers.bcrypt [None req-f8501560-b4a0-4451-b059-f747861f30a0 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=81758) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 24 17:26:40.564216 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG passlib.handlers.bcrypt [None req-f8501560-b4a0-4451-b059-f747861f30a0 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81758) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 24 17:26:40.575196 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 6/12] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:26:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 17:26:40.610003 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-f8501560-b4a0-4451-b059-f747861f30a0 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:26:40.611572 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-f8501560-b4a0-4451-b059-f747861f30a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:26:40.707606 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 7/13] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:26:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 17:26:40.846593 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f875a457-a340-42d3-a3b2-0ebfdc90ee0f None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:40.870494 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:26:40.956633 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f875a457-a340-42d3-a3b2-0ebfdc90ee0f None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:40.972700 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8501560-b4a0-4451-b059-f747861f30a0 None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:40.991382 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36373f99-ba10-489d-9c7c-f972fa7b1a8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:40.992024 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36373f99-ba10-489d-9c7c-f972fa7b1a8b None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:40.992307 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36373f99-ba10-489d-9c7c-f972fa7b1a8b None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:40.992621 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36373f99-ba10-489d-9c7c-f972fa7b1a8b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:41.003735 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36373f99-ba10-489d-9c7c-f972fa7b1a8b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:26:41.012672 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application [None req-36373f99-ba10-489d-9c7c-f972fa7b1a8b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:26:41.012672 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:26:41.012672 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:26:41.012672 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:26:41.012672 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:26:41.012672 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:26:41.012672 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:26:41.012672 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:26:41.012672 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:26:41.012672 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:26:41.012672 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:26:41.012672 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:26:41.012672 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:26:41.012672 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:26:41.012672 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:26:41.012672 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:26:41.013760 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:26:41.013760 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:26:41.013760 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:26:41.013760 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:26:41.013760 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:26:41.013760 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:26:41.013760 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:26:41.013760 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:26:41.013760 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:26:41.013760 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:26:41.013760 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:26:41.013760 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application with Lock( Aug 24 17:26:41.013760 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:26:41.013760 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:26:41.013760 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:26:41.013760 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:26:41.014581 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:26:41.014581 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:26:41.014581 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:26:41.014581 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:26:41.014581 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:26:41.014581 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:26:41.014581 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:26:41.014581 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:26:41.014581 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:26:41.014581 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:26:41.014581 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:26:41.014581 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Aug 24 17:26:41.014581 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 7/14] 10.222.0.45 () {56 vars in 1096 bytes} [Wed Aug 24 17:26:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:26:41.019435 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36373f99-ba10-489d-9c7c-f972fa7b1a8b None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:41.073706 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8501560-b4a0-4451-b059-f747861f30a0 None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:41.075906 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:26:41.104355 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:26:41.111583 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09af7e9d-ea6f-49fb-abe9-d1ae5154cced None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:41.112436 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09af7e9d-ea6f-49fb-abe9-d1ae5154cced None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:41.112755 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09af7e9d-ea6f-49fb-abe9-d1ae5154cced None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:41.113032 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09af7e9d-ea6f-49fb-abe9-d1ae5154cced None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:41.122467 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36373f99-ba10-489d-9c7c-f972fa7b1a8b None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:41.155292 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e30e9c-c027-4f26-a7bb-4a7094b60c74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:41.155890 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e30e9c-c027-4f26-a7bb-4a7094b60c74 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:41.156168 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e30e9c-c027-4f26-a7bb-4a7094b60c74 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:41.156441 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e30e9c-c027-4f26-a7bb-4a7094b60c74 None admin] PATH_INFO: `/v3/roles` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:41.157138 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e30e9c-c027-4f26-a7bb-4a7094b60c74 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:26:41.157443 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e30e9c-c027-4f26-a7bb-4a7094b60c74 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:41.158836 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e30e9c-c027-4f26-a7bb-4a7094b60c74 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:41.170699 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 8/15] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Aug 24 17:26:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:26:41.176143 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e30e9c-c027-4f26-a7bb-4a7094b60c74 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:41.178696 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09af7e9d-ea6f-49fb-abe9-d1ae5154cced None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:41.180218 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09af7e9d-ea6f-49fb-abe9-d1ae5154cced None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:41.272792 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e30e9c-c027-4f26-a7bb-4a7094b60c74 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:41.276286 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 8/16] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Aug 24 17:26:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 17:26:41.281826 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09af7e9d-ea6f-49fb-abe9-d1ae5154cced None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:41.284455 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:26:41.306488 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4290f94-f629-4317-a3ab-2d4976ffe03c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:41.307610 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4290f94-f629-4317-a3ab-2d4976ffe03c None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:41.307900 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4290f94-f629-4317-a3ab-2d4976ffe03c None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:41.308179 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4290f94-f629-4317-a3ab-2d4976ffe03c None admin] PATH_INFO: `/v3/users/cc2669e1aa494dbcb7164d2e5e46378c` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:41.309366 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4290f94-f629-4317-a3ab-2d4976ffe03c None admin] RBAC: Authorizing `identity:get_user(user_id=cc2669e1aa494dbcb7164d2e5e46378c)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:41.310833 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4290f94-f629-4317-a3ab-2d4976ffe03c None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:41.312434 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 9/17] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Aug 24 17:26:41 2022] GET /identity/v3/users/cc2669e1aa494dbcb7164d2e5e46378c => generated 247 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:26:41.317842 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4290f94-f629-4317-a3ab-2d4976ffe03c None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:41.331876 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:26:41.382404 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09af7e9d-ea6f-49fb-abe9-d1ae5154cced None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:41.415695 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c33be4-5be9-4344-9805-19746788e6b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:41.417111 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c33be4-5be9-4344-9805-19746788e6b5 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:41.417218 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4290f94-f629-4317-a3ab-2d4976ffe03c None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:41.417407 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c33be4-5be9-4344-9805-19746788e6b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:41.417663 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c33be4-5be9-4344-9805-19746788e6b5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:41.436048 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c33be4-5be9-4344-9805-19746788e6b5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:41.437553 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c33be4-5be9-4344-9805-19746788e6b5 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:41.451791 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f436ab3-1344-4b7e-9bcd-7082f89a4933 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:41.452423 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f436ab3-1344-4b7e-9bcd-7082f89a4933 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:41.452732 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f436ab3-1344-4b7e-9bcd-7082f89a4933 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:41.453014 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f436ab3-1344-4b7e-9bcd-7082f89a4933 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:41.454136 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f436ab3-1344-4b7e-9bcd-7082f89a4933 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:41.455489 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application [None req-83c33be4-5be9-4344-9805-19746788e6b5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:26:41.455489 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:26:41.455489 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:26:41.455489 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:26:41.455489 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:26:41.455489 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:26:41.455489 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:26:41.455489 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:26:41.455489 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:26:41.455489 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:26:41.455489 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:26:41.455489 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:26:41.455489 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:26:41.455489 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:26:41.455489 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:26:41.455489 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:26:41.456456 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:26:41.456456 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:26:41.456456 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:26:41.456456 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:26:41.456456 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:26:41.456456 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:26:41.456456 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:26:41.456456 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application with Lock( Aug 24 17:26:41.456456 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:26:41.456456 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:26:41.456456 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:26:41.456456 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:26:41.456456 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:26:41.456456 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:26:41.456456 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:26:41.456456 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:26:41.456456 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:26:41.457347 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:26:41.457347 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:26:41.457347 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Aug 24 17:26:41.457514 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f436ab3-1344-4b7e-9bcd-7082f89a4933 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:41.457577 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 9/18] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 17:26:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 17:26:41.457643 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 10/19] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 17:26:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:26:41.462456 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c33be4-5be9-4344-9805-19746788e6b5 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:41.462641 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f436ab3-1344-4b7e-9bcd-7082f89a4933 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:41.518475 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:26:41.542485 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:26:41.561352 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f436ab3-1344-4b7e-9bcd-7082f89a4933 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:41.562818 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c33be4-5be9-4344-9805-19746788e6b5 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:41.594868 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506301e0-37ab-4d27-addc-3cc6ef9b88c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:41.595251 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a4bbad-b1fe-446e-89aa-b8e8d84afc39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:41.595408 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506301e0-37ab-4d27-addc-3cc6ef9b88c0 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:41.595685 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506301e0-37ab-4d27-addc-3cc6ef9b88c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:41.595851 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a4bbad-b1fe-446e-89aa-b8e8d84afc39 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:41.595951 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506301e0-37ab-4d27-addc-3cc6ef9b88c0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:41.596133 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a4bbad-b1fe-446e-89aa-b8e8d84afc39 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:41.596404 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a4bbad-b1fe-446e-89aa-b8e8d84afc39 None admin] PATH_INFO: `/v3/domains` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:41.596705 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506301e0-37ab-4d27-addc-3cc6ef9b88c0 None admin] RBAC: Adding query filter params (role.id=e411ad9eb96e4e8d85b59b5b700ada7b, scope.domain.id=default, user.id=cc2669e1aa494dbcb7164d2e5e46378c) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:26:41.597009 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506301e0-37ab-4d27-addc-3cc6ef9b88c0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:41.597095 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a4bbad-b1fe-446e-89aa-b8e8d84afc39 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:26:41.597361 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a4bbad-b1fe-446e-89aa-b8e8d84afc39 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:41.598453 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506301e0-37ab-4d27-addc-3cc6ef9b88c0 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:41.598720 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a4bbad-b1fe-446e-89aa-b8e8d84afc39 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:41.605902 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 11/20] 10.222.0.45 () {56 vars in 1322 bytes} [Wed Aug 24 17:26:41 2022] GET /identity/v3/role_assignments?user.id=cc2669e1aa494dbcb7164d2e5e46378c&scope.domain.id=default&role.id=e411ad9eb96e4e8d85b59b5b700ada7b => generated 233 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:26:41.619176 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 10/21] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 24 17:26:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:26:43.675829 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0657fbff-122e-415f-a0a4-1d66bf7eb2e6 None None] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:43.676888 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0657fbff-122e-415f-a0a4-1d66bf7eb2e6 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:43.677397 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0657fbff-122e-415f-a0a4-1d66bf7eb2e6 None None] PATH_INFO: `/` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:43.679210 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 12/22] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:26:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:26:43.684765 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90cbb48-8508-4eba-ad97-a9521c3bb920 None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:43.685577 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90cbb48-8508-4eba-ad97-a9521c3bb920 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:43.686057 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90cbb48-8508-4eba-ad97-a9521c3bb920 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:43.714528 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-b90cbb48-8508-4eba-ad97-a9521c3bb920 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:26:43.716076 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-b90cbb48-8508-4eba-ad97-a9521c3bb920 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:26:43.807061 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 11/23] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:26:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:26:43.926537 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9617b800-62c1-4605-8f88-652fcfc6e566 None None] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:43.927125 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9617b800-62c1-4605-8f88-652fcfc6e566 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:43.927640 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9617b800-62c1-4605-8f88-652fcfc6e566 None None] PATH_INFO: `/` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:43.929476 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 13/24] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:26:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:26:43.934970 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3146b0d-e904-42ca-b130-ccf3793357eb None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:43.935514 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3146b0d-e904-42ca-b130-ccf3793357eb None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:43.935993 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3146b0d-e904-42ca-b130-ccf3793357eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:43.965622 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-b3146b0d-e904-42ca-b130-ccf3793357eb None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:26:43.967542 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-b3146b0d-e904-42ca-b130-ccf3793357eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:26:44.021304 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:26:44.054361 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 12/25] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:26:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:26:44.179212 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92764999-0327-4c93-89c1-466907a1a089 None None] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:44.179547 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92764999-0327-4c93-89c1-466907a1a089 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:44.179733 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92764999-0327-4c93-89c1-466907a1a089 None None] PATH_INFO: `/` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:44.181330 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 14/26] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:26:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:26:44.187083 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8563ba-2dce-43b2-88dd-db20ebd564fa None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:44.187379 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8563ba-2dce-43b2-88dd-db20ebd564fa None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:44.187665 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8563ba-2dce-43b2-88dd-db20ebd564fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:44.209949 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-8a8563ba-2dce-43b2-88dd-db20ebd564fa None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:26:44.211484 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-8a8563ba-2dce-43b2-88dd-db20ebd564fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:26:44.282021 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 13/27] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:26:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:26:44.300021 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92764999-0327-4c93-89c1-466907a1a089 None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:44.431816 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92764999-0327-4c93-89c1-466907a1a089 None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:44.460951 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8563ba-2dce-43b2-88dd-db20ebd564fa None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:44.467390 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf0e82b-b9f2-46fe-a1b4-c7cf45911e2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:44.468040 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf0e82b-b9f2-46fe-a1b4-c7cf45911e2a None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:44.468324 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf0e82b-b9f2-46fe-a1b4-c7cf45911e2a None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:44.468619 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf0e82b-b9f2-46fe-a1b4-c7cf45911e2a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:44.480195 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf0e82b-b9f2-46fe-a1b4-c7cf45911e2a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:26:44.485368 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application [None req-bdf0e82b-b9f2-46fe-a1b4-c7cf45911e2a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:26:44.485368 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:26:44.485368 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:26:44.485368 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:26:44.485368 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:26:44.485368 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:26:44.485368 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:26:44.485368 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:26:44.485368 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:26:44.485368 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:26:44.485368 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:26:44.485368 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:26:44.485368 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:26:44.485368 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:26:44.485368 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:26:44.485368 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:26:44.486191 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:26:44.486191 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:26:44.486191 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:26:44.486191 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:26:44.486191 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:26:44.486191 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:26:44.486191 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:26:44.486191 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:26:44.486191 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:26:44.486191 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:26:44.486191 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:26:44.486191 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application with Lock( Aug 24 17:26:44.486191 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:26:44.486191 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:26:44.486191 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:26:44.486191 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:26:44.486933 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:26:44.486933 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:26:44.486933 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:26:44.486933 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:26:44.486933 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:26:44.486933 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:26:44.486933 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:26:44.486933 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:26:44.486933 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:26:44.486933 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:26:44.486933 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:26:44.486933 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Aug 24 17:26:44.486933 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 15/28] 10.222.0.45 () {56 vars in 1096 bytes} [Wed Aug 24 17:26:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:26:44.494344 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf0e82b-b9f2-46fe-a1b4-c7cf45911e2a None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:44.503239 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:26:44.664148 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8563ba-2dce-43b2-88dd-db20ebd564fa None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:44.672493 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:26:44.685547 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf0e82b-b9f2-46fe-a1b4-c7cf45911e2a None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:44.722122 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f444f88-a692-44f1-bb14-615f1749e33a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:44.722743 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f444f88-a692-44f1-bb14-615f1749e33a None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:44.723072 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f444f88-a692-44f1-bb14-615f1749e33a None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:44.723296 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f444f88-a692-44f1-bb14-615f1749e33a None admin] PATH_INFO: `/v3/roles` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:44.724177 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f444f88-a692-44f1-bb14-615f1749e33a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:26:44.724484 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f444f88-a692-44f1-bb14-615f1749e33a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:44.725052 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3164956f-6f94-4331-a8e6-d1c5b266a69e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:44.725685 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3164956f-6f94-4331-a8e6-d1c5b266a69e None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:44.725917 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f444f88-a692-44f1-bb14-615f1749e33a None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:44.726014 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3164956f-6f94-4331-a8e6-d1c5b266a69e None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:44.726224 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3164956f-6f94-4331-a8e6-d1c5b266a69e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:44.734481 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 16/29] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Aug 24 17:26:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:26:44.735471 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463430d5-fa4d-46ee-964e-9f81faba1297 None None] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:44.735758 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463430d5-fa4d-46ee-964e-9f81faba1297 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:44.736076 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463430d5-fa4d-46ee-964e-9f81faba1297 None None] PATH_INFO: `/` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:44.737067 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3164956f-6f94-4331-a8e6-d1c5b266a69e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:44.737412 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 17/30] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:26:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:26:44.738092 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463430d5-fa4d-46ee-964e-9f81faba1297 None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:44.738896 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3164956f-6f94-4331-a8e6-d1c5b266a69e None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:44.754647 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application [None req-3164956f-6f94-4331-a8e6-d1c5b266a69e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:26:44.754647 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:26:44.754647 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:26:44.754647 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:26:44.754647 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:26:44.754647 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:26:44.754647 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:26:44.754647 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:26:44.754647 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:26:44.754647 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:26:44.754647 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:26:44.754647 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:26:44.754647 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:26:44.754647 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:26:44.754647 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:26:44.754647 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:26:44.755515 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:26:44.755515 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:26:44.755515 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:26:44.755515 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:26:44.755515 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:26:44.755515 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:26:44.755515 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:26:44.755515 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application with Lock( Aug 24 17:26:44.755515 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:26:44.755515 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:26:44.755515 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:26:44.755515 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:26:44.755515 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:26:44.755515 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:26:44.755515 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:26:44.755515 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:26:44.755515 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:26:44.756424 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:26:44.756424 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:26:44.756424 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Aug 24 17:26:44.756424 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 14/31] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 17:26:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:26:44.757327 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce2654e-c316-4df5-9774-888bc2fd9aaa None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:44.757487 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce2654e-c316-4df5-9774-888bc2fd9aaa None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:44.758358 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce2654e-c316-4df5-9774-888bc2fd9aaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:44.781677 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-cce2654e-c316-4df5-9774-888bc2fd9aaa None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:26:44.782983 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-cce2654e-c316-4df5-9774-888bc2fd9aaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:26:44.820579 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:26:44.860875 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463430d5-fa4d-46ee-964e-9f81faba1297 None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:44.861230 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 15/32] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:26:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:26:44.862183 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce2654e-c316-4df5-9774-888bc2fd9aaa None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:44.897332 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9101b070-5594-49ac-ae39-20d3afcd7162 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:44.898294 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9101b070-5594-49ac-ae39-20d3afcd7162 None admin] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:44.898770 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9101b070-5594-49ac-ae39-20d3afcd7162 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:44.899293 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9101b070-5594-49ac-ae39-20d3afcd7162 None admin] PATH_INFO: `/v3/roles` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:44.900183 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9101b070-5594-49ac-ae39-20d3afcd7162 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:44.902404 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9101b070-5594-49ac-ae39-20d3afcd7162 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:44.956545 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:26:44.971684 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce2654e-c316-4df5-9774-888bc2fd9aaa None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:45.007395 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefe724b-8dad-4222-ba95-2f58d76dbda4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:45.008214 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefe724b-8dad-4222-ba95-2f58d76dbda4 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:45.008579 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefe724b-8dad-4222-ba95-2f58d76dbda4 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:45.008864 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefe724b-8dad-4222-ba95-2f58d76dbda4 None admin] PATH_INFO: `/v3/users/cc2669e1aa494dbcb7164d2e5e46378c` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:45.010985 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefe724b-8dad-4222-ba95-2f58d76dbda4 None admin] RBAC: Authorizing `identity:get_user(user_id=cc2669e1aa494dbcb7164d2e5e46378c)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:45.012497 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefe724b-8dad-4222-ba95-2f58d76dbda4 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:45.014425 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 16/33] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Aug 24 17:26:44 2022] GET /identity/v3/users/cc2669e1aa494dbcb7164d2e5e46378c => generated 247 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:26:45.018894 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefe724b-8dad-4222-ba95-2f58d76dbda4 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:45.056465 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 18/34] 10.222.0.45 () {60 vars in 1131 bytes} [Wed Aug 24 17:26:44 2022] POST /identity/v3/roles => generated 219 bytes in 318 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:26:45.058120 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9101b070-5594-49ac-ae39-20d3afcd7162 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:45.180410 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefe724b-8dad-4222-ba95-2f58d76dbda4 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:45.221465 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9101b070-5594-49ac-ae39-20d3afcd7162 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:45.224930 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7011a805-7249-412c-8329-8475362472a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:45.225845 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7011a805-7249-412c-8329-8475362472a7 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:45.226327 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7011a805-7249-412c-8329-8475362472a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:45.227056 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7011a805-7249-412c-8329-8475362472a7 None admin] PATH_INFO: `/v3/domains` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:45.227944 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7011a805-7249-412c-8329-8475362472a7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:26:45.228443 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7011a805-7249-412c-8329-8475362472a7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:45.230102 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7011a805-7249-412c-8329-8475362472a7 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:45.243785 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 17/35] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 24 17:26:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:26:45.250951 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7011a805-7249-412c-8329-8475362472a7 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:45.284896 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:26:45.286786 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:26:45.300186 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd58d7c-e054-413e-9d09-3c8fe3b4a86d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:45.300880 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd58d7c-e054-413e-9d09-3c8fe3b4a86d None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:45.301160 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd58d7c-e054-413e-9d09-3c8fe3b4a86d None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:45.301435 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd58d7c-e054-413e-9d09-3c8fe3b4a86d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:45.303399 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd58d7c-e054-413e-9d09-3c8fe3b4a86d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:45.305030 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd58d7c-e054-413e-9d09-3c8fe3b4a86d None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:45.311337 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 19/36] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 17:26:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:26:45.314325 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd58d7c-e054-413e-9d09-3c8fe3b4a86d None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:45.405670 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7011a805-7249-412c-8329-8475362472a7 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:45.456335 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9bdb28-7c9a-4136-b0b5-9f8b03ef4db9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:45.456977 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9bdb28-7c9a-4136-b0b5-9f8b03ef4db9 None admin] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:45.457256 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9bdb28-7c9a-4136-b0b5-9f8b03ef4db9 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:45.457526 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9bdb28-7c9a-4136-b0b5-9f8b03ef4db9 None admin] PATH_INFO: `/v3/projects` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:45.458173 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9bdb28-7c9a-4136-b0b5-9f8b03ef4db9 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:45.459606 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9bdb28-7c9a-4136-b0b5-9f8b03ef4db9 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:45.464755 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd58d7c-e054-413e-9d09-3c8fe3b4a86d None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:45.509367 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56863dea-7ead-4334-9afc-4b8cb920f6a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:45.510754 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56863dea-7ead-4334-9afc-4b8cb920f6a5 None admin] REQUEST_METHOD: `PUT` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:45.511257 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56863dea-7ead-4334-9afc-4b8cb920f6a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:45.511753 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56863dea-7ead-4334-9afc-4b8cb920f6a5 None admin] PATH_INFO: `/v3/domains/default/users/cc2669e1aa494dbcb7164d2e5e46378c/roles/e411ad9eb96e4e8d85b59b5b700ada7b` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:45.514062 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56863dea-7ead-4334-9afc-4b8cb920f6a5 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=cc2669e1aa494dbcb7164d2e5e46378c, role_id=e411ad9eb96e4e8d85b59b5b700ada7b)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:45.515713 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56863dea-7ead-4334-9afc-4b8cb920f6a5 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:45.542561 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 20/37] 10.222.0.45 () {58 vars in 1361 bytes} [Wed Aug 24 17:26:45 2022] PUT /identity/v3/domains/default/users/cc2669e1aa494dbcb7164d2e5e46378c/roles/e411ad9eb96e4e8d85b59b5b700ada7b => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:26:45.543384 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56863dea-7ead-4334-9afc-4b8cb920f6a5 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:45.616286 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 18/38] 10.222.0.45 () {60 vars in 1141 bytes} [Wed Aug 24 17:26:45 2022] POST /identity/v3/projects => generated 301 bytes in 366 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:26:45.616844 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:26:45.618170 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec01b52b-f397-4139-8154-dacb4f37108a None None] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:45.618466 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec01b52b-f397-4139-8154-dacb4f37108a None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:45.618748 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec01b52b-f397-4139-8154-dacb4f37108a None None] PATH_INFO: `/` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:45.621491 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 19/39] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:26:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:26:45.629317 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1a12e9-659b-4772-9ea8-83285715d691 None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:45.629945 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1a12e9-659b-4772-9ea8-83285715d691 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:45.630394 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1a12e9-659b-4772-9ea8-83285715d691 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:45.661224 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-ee1a12e9-659b-4772-9ea8-83285715d691 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:26:45.663352 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-ee1a12e9-659b-4772-9ea8-83285715d691 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:26:45.675815 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56863dea-7ead-4334-9afc-4b8cb920f6a5 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:45.726094 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a8fcc4-4171-440a-893b-e28c0e0cabb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:45.727021 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a8fcc4-4171-440a-893b-e28c0e0cabb0 None admin] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:45.727253 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a8fcc4-4171-440a-893b-e28c0e0cabb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:45.727527 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a8fcc4-4171-440a-893b-e28c0e0cabb0 None admin] PATH_INFO: `/v3/roles` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:45.728234 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a8fcc4-4171-440a-893b-e28c0e0cabb0 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:45.730881 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a8fcc4-4171-440a-893b-e28c0e0cabb0 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:45.743793 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:26:45.751612 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 21/40] 10.222.0.45 () {60 vars in 1131 bytes} [Wed Aug 24 17:26:45 2022] POST /identity/v3/roles => generated 225 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:26:45.754708 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 20/41] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:26:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:26:45.754708 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d432ce08-2808-4770-a751-1efda77edf6e None None] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:45.754708 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d432ce08-2808-4770-a751-1efda77edf6e None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:45.754708 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d432ce08-2808-4770-a751-1efda77edf6e None None] PATH_INFO: `/` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:45.755557 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 21/42] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:26:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:26:45.770871 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1efbff-db3f-427f-8e55-1b0c273a489f None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:45.771145 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1efbff-db3f-427f-8e55-1b0c273a489f None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:45.771402 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1efbff-db3f-427f-8e55-1b0c273a489f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:45.794880 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-4a1efbff-db3f-427f-8e55-1b0c273a489f None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:26:45.796188 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-4a1efbff-db3f-427f-8e55-1b0c273a489f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:26:45.880158 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 22/43] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:26:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:26:46.047765 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a8fcc4-4171-440a-893b-e28c0e0cabb0 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:46.148164 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a8fcc4-4171-440a-893b-e28c0e0cabb0 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:46.148567 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1efbff-db3f-427f-8e55-1b0c273a489f None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:46.166673 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:26:46.188757 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597cdb51-bceb-4f76-9e28-4b9dff98dd78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:46.189401 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597cdb51-bceb-4f76-9e28-4b9dff98dd78 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:46.189694 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597cdb51-bceb-4f76-9e28-4b9dff98dd78 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:46.189973 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597cdb51-bceb-4f76-9e28-4b9dff98dd78 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:46.191105 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597cdb51-bceb-4f76-9e28-4b9dff98dd78 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:46.192596 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597cdb51-bceb-4f76-9e28-4b9dff98dd78 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:46.194183 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 22/44] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 17:26:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:26:46.200076 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597cdb51-bceb-4f76-9e28-4b9dff98dd78 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:46.254137 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1efbff-db3f-427f-8e55-1b0c273a489f None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:46.282884 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:26:46.291536 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83aa09ac-8ce4-42a3-993e-bd42357336a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:46.292191 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83aa09ac-8ce4-42a3-993e-bd42357336a5 None admin] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:46.292475 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83aa09ac-8ce4-42a3-993e-bd42357336a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:46.292793 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83aa09ac-8ce4-42a3-993e-bd42357336a5 None admin] PATH_INFO: `/v3/roles` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:46.293478 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83aa09ac-8ce4-42a3-993e-bd42357336a5 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:46.295001 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83aa09ac-8ce4-42a3-993e-bd42357336a5 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:46.305608 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597cdb51-bceb-4f76-9e28-4b9dff98dd78 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:46.309982 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 23/45] 10.222.0.45 () {60 vars in 1131 bytes} [Wed Aug 24 17:26:46 2022] POST /identity/v3/roles => generated 223 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:26:46.349708 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21085376-b31d-465f-bcbd-82db080e54da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:46.350345 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21085376-b31d-465f-bcbd-82db080e54da None admin] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:46.350692 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21085376-b31d-465f-bcbd-82db080e54da None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:46.350971 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21085376-b31d-465f-bcbd-82db080e54da None admin] PATH_INFO: `/v3/projects` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:46.351960 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21085376-b31d-465f-bcbd-82db080e54da None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:46.353520 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21085376-b31d-465f-bcbd-82db080e54da None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:46.396581 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 23/46] 10.222.0.45 () {60 vars in 1141 bytes} [Wed Aug 24 17:26:46 2022] POST /identity/v3/projects => generated 312 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:26:47.601373 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d2a24f-ace7-4196-a578-2a5ae8e7f0c3 None None] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:47.602143 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d2a24f-ace7-4196-a578-2a5ae8e7f0c3 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:47.602225 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d2a24f-ace7-4196-a578-2a5ae8e7f0c3 None None] PATH_INFO: `/` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:47.603519 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 24/47] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:26:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:26:47.609585 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64aec947-8262-4cc2-afe1-7a5d09194cc7 None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:47.610012 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64aec947-8262-4cc2-afe1-7a5d09194cc7 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:47.610149 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64aec947-8262-4cc2-afe1-7a5d09194cc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:47.628413 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-64aec947-8262-4cc2-afe1-7a5d09194cc7 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:26:47.629960 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-64aec947-8262-4cc2-afe1-7a5d09194cc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:26:47.656540 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:26:47.696832 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 24/48] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:26:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:26:47.965534 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d2a24f-ace7-4196-a578-2a5ae8e7f0c3 None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:48.066152 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d2a24f-ace7-4196-a578-2a5ae8e7f0c3 None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:48.098764 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53448f9b-b6c2-4aa5-aca1-0086519d9fd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:48.099394 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53448f9b-b6c2-4aa5-aca1-0086519d9fd6 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:48.099682 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53448f9b-b6c2-4aa5-aca1-0086519d9fd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:48.099959 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53448f9b-b6c2-4aa5-aca1-0086519d9fd6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:48.112172 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53448f9b-b6c2-4aa5-aca1-0086519d9fd6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:26:48.118039 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application [None req-53448f9b-b6c2-4aa5-aca1-0086519d9fd6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:26:48.118039 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:26:48.118039 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:26:48.118039 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:26:48.118039 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:26:48.118039 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:26:48.118039 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:26:48.118039 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:26:48.118039 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:26:48.118039 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:26:48.118039 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:26:48.118039 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:26:48.118039 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:26:48.118039 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:26:48.118039 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:26:48.118039 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:26:48.119016 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:26:48.119016 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:26:48.119016 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:26:48.119016 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:26:48.119016 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:26:48.119016 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:26:48.119016 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:26:48.119016 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:26:48.119016 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:26:48.119016 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:26:48.119016 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:26:48.119016 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application with Lock( Aug 24 17:26:48.119016 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:26:48.119016 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:26:48.119016 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:26:48.119016 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:26:48.120065 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:26:48.120065 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:26:48.120065 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:26:48.120065 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:26:48.120065 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:26:48.120065 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:26:48.120065 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:26:48.120065 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:26:48.120065 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:26:48.120065 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:26:48.120065 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:26:48.120065 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Aug 24 17:26:48.120065 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 25/49] 10.222.0.45 () {56 vars in 1096 bytes} [Wed Aug 24 17:26:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:26:48.127309 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64aec947-8262-4cc2-afe1-7a5d09194cc7 None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:48.277154 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c79464-c72e-4410-b6f5-dad9d3719c3b None None] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:48.277476 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c79464-c72e-4410-b6f5-dad9d3719c3b None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:48.277763 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c79464-c72e-4410-b6f5-dad9d3719c3b None None] PATH_INFO: `/` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:48.279117 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 26/50] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:26:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 17:26:48.284189 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d56c95-8e96-49f0-aebb-71114257abad None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:48.284488 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d56c95-8e96-49f0-aebb-71114257abad None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:48.284804 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d56c95-8e96-49f0-aebb-71114257abad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:48.302861 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-b1d56c95-8e96-49f0-aebb-71114257abad None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:26:48.304083 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-b1d56c95-8e96-49f0-aebb-71114257abad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:26:48.330522 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64aec947-8262-4cc2-afe1-7a5d09194cc7 None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:48.347127 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:26:48.364362 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf35e55a-862c-42b9-900c-50f814f0d874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:48.365012 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf35e55a-862c-42b9-900c-50f814f0d874 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:48.365297 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf35e55a-862c-42b9-900c-50f814f0d874 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:48.365579 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf35e55a-862c-42b9-900c-50f814f0d874 None admin] PATH_INFO: `/v3/roles` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:48.367137 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf35e55a-862c-42b9-900c-50f814f0d874 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:26:48.367455 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf35e55a-862c-42b9-900c-50f814f0d874 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:48.368923 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf35e55a-862c-42b9-900c-50f814f0d874 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:48.373447 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 27/51] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:26:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 17:26:48.377571 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 25/52] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Aug 24 17:26:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:26:48.383128 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d56c95-8e96-49f0-aebb-71114257abad None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:48.479076 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d56c95-8e96-49f0-aebb-71114257abad None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:48.512719 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5e0f10-ce69-48f3-af04-4a22c8e63092 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:48.513344 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5e0f10-ce69-48f3-af04-4a22c8e63092 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:48.513631 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5e0f10-ce69-48f3-af04-4a22c8e63092 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:48.513909 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5e0f10-ce69-48f3-af04-4a22c8e63092 None admin] PATH_INFO: `/v3/users/cc2669e1aa494dbcb7164d2e5e46378c` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:48.515037 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5e0f10-ce69-48f3-af04-4a22c8e63092 None admin] RBAC: Authorizing `identity:get_user(user_id=cc2669e1aa494dbcb7164d2e5e46378c)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:48.516496 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5e0f10-ce69-48f3-af04-4a22c8e63092 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:48.518073 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 28/53] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Aug 24 17:26:48 2022] GET /identity/v3/users/cc2669e1aa494dbcb7164d2e5e46378c => generated 247 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:26:48.523805 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf35e55a-862c-42b9-900c-50f814f0d874 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:48.551458 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:26:48.622379 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf35e55a-862c-42b9-900c-50f814f0d874 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:48.641596 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5e0f10-ce69-48f3-af04-4a22c8e63092 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:48.655647 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec26a80e-9a3a-497c-83d4-ea04598ab98d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:48.656303 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec26a80e-9a3a-497c-83d4-ea04598ab98d None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:48.656611 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec26a80e-9a3a-497c-83d4-ea04598ab98d None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:48.656898 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec26a80e-9a3a-497c-83d4-ea04598ab98d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:48.658031 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec26a80e-9a3a-497c-83d4-ea04598ab98d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:48.659455 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec26a80e-9a3a-497c-83d4-ea04598ab98d None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:48.661074 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 26/54] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 17:26:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:26:48.667513 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec26a80e-9a3a-497c-83d4-ea04598ab98d None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:48.687751 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:26:48.740651 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5e0f10-ce69-48f3-af04-4a22c8e63092 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:48.759568 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:26:48.766700 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec26a80e-9a3a-497c-83d4-ea04598ab98d None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:48.776280 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e684a6f5-d4bb-418a-8f92-2cd77a93db79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:48.776950 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e684a6f5-d4bb-418a-8f92-2cd77a93db79 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:48.777231 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e684a6f5-d4bb-418a-8f92-2cd77a93db79 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:48.777501 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e684a6f5-d4bb-418a-8f92-2cd77a93db79 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:48.778624 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e684a6f5-d4bb-418a-8f92-2cd77a93db79 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:48.780067 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e684a6f5-d4bb-418a-8f92-2cd77a93db79 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:48.781658 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 29/55] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 17:26:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:26:48.786991 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e684a6f5-d4bb-418a-8f92-2cd77a93db79 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:48.803335 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb73441-f07f-4675-9c51-5c2510f5bcca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:48.803888 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb73441-f07f-4675-9c51-5c2510f5bcca None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:48.804166 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb73441-f07f-4675-9c51-5c2510f5bcca None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:48.804441 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb73441-f07f-4675-9c51-5c2510f5bcca None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:48.805206 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb73441-f07f-4675-9c51-5c2510f5bcca None admin] RBAC: Adding query filter params (role.id=e411ad9eb96e4e8d85b59b5b700ada7b, scope.domain.id=default, user.id=cc2669e1aa494dbcb7164d2e5e46378c) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:26:48.805512 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb73441-f07f-4675-9c51-5c2510f5bcca None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:48.806949 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb73441-f07f-4675-9c51-5c2510f5bcca None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:48.816582 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 27/56] 10.222.0.45 () {56 vars in 1322 bytes} [Wed Aug 24 17:26:48 2022] GET /identity/v3/role_assignments?user.id=cc2669e1aa494dbcb7164d2e5e46378c&scope.domain.id=default&role.id=e411ad9eb96e4e8d85b59b5b700ada7b => generated 530 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:26:48.891100 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e684a6f5-d4bb-418a-8f92-2cd77a93db79 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:48.906410 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:26:48.927870 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a812d59-5c2d-4332-aac0-b858a67d8ad5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:48.928494 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a812d59-5c2d-4332-aac0-b858a67d8ad5 None admin] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:48.928802 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a812d59-5c2d-4332-aac0-b858a67d8ad5 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:48.929073 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a812d59-5c2d-4332-aac0-b858a67d8ad5 None admin] PATH_INFO: `/v3/projects` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:48.929724 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a812d59-5c2d-4332-aac0-b858a67d8ad5 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:48.931180 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a812d59-5c2d-4332-aac0-b858a67d8ad5 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:48.958985 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 30/57] 10.222.0.45 () {60 vars in 1140 bytes} [Wed Aug 24 17:26:48 2022] POST /identity/v3/projects => generated 298 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:26:50.825459 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e31eaf-7c7a-4336-ba6d-6c41644fdcd2 None None] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:50.825887 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e31eaf-7c7a-4336-ba6d-6c41644fdcd2 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:50.825997 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e31eaf-7c7a-4336-ba6d-6c41644fdcd2 None None] PATH_INFO: `/` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:50.827385 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 28/58] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:26:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:26:50.832582 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd6a95c-4cf8-4ecc-9506-3b746557e970 None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:50.832967 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd6a95c-4cf8-4ecc-9506-3b746557e970 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:50.833130 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd6a95c-4cf8-4ecc-9506-3b746557e970 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:50.851171 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-0fd6a95c-4cf8-4ecc-9506-3b746557e970 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:26:50.852653 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-0fd6a95c-4cf8-4ecc-9506-3b746557e970 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:26:50.917251 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 31/59] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:26:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:26:51.194764 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e31eaf-7c7a-4336-ba6d-6c41644fdcd2 None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:51.292596 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e31eaf-7c7a-4336-ba6d-6c41644fdcd2 None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:51.324571 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86605bb-e6a0-4057-91c8-1c31963c3274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:51.325208 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86605bb-e6a0-4057-91c8-1c31963c3274 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:51.325490 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86605bb-e6a0-4057-91c8-1c31963c3274 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:51.325770 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86605bb-e6a0-4057-91c8-1c31963c3274 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:51.326874 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86605bb-e6a0-4057-91c8-1c31963c3274 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:51.328311 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86605bb-e6a0-4057-91c8-1c31963c3274 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:51.329856 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 29/60] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 17:26:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:26:51.335316 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd6a95c-4cf8-4ecc-9506-3b746557e970 None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:51.426266 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:26:51.433243 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd6a95c-4cf8-4ecc-9506-3b746557e970 None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:51.466114 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b8bb0b-6d93-4c48-990e-c519ce03550e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:51.466660 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b8bb0b-6d93-4c48-990e-c519ce03550e None admin] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:51.466945 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b8bb0b-6d93-4c48-990e-c519ce03550e None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:51.467221 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b8bb0b-6d93-4c48-990e-c519ce03550e None admin] PATH_INFO: `/v3/users` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:51.467886 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b8bb0b-6d93-4c48-990e-c519ce03550e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:51.469361 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b8bb0b-6d93-4c48-990e-c519ce03550e None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:51.495902 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 32/61] 10.222.0.45 () {60 vars in 1132 bytes} [Wed Aug 24 17:26:51 2022] POST /identity/v3/users => generated 275 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:26:54.470016 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d981ff78-c7eb-4bc0-90cc-457fde4b21b5 None None] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:54.471050 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d981ff78-c7eb-4bc0-90cc-457fde4b21b5 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:54.471589 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d981ff78-c7eb-4bc0-90cc-457fde4b21b5 None None] PATH_INFO: `/` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:54.473163 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 30/62] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:26:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:26:54.481058 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16064938-b399-49f3-9b36-39b97687ce87 None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:54.481520 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16064938-b399-49f3-9b36-39b97687ce87 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:54.481605 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16064938-b399-49f3-9b36-39b97687ce87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:54.509675 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-16064938-b399-49f3-9b36-39b97687ce87 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:26:54.511413 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-16064938-b399-49f3-9b36-39b97687ce87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:26:54.593318 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98d14da-2f32-4cf4-beda-796809f71f99 None None] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:54.593961 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98d14da-2f32-4cf4-beda-796809f71f99 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:54.594442 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98d14da-2f32-4cf4-beda-796809f71f99 None None] PATH_INFO: `/` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:54.595986 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 31/63] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:26:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 17:26:54.603038 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 33/64] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:26:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:26:54.603450 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9040148-a5d8-45ca-9087-22fcc19c998c None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:54.604269 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9040148-a5d8-45ca-9087-22fcc19c998c None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:54.604627 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9040148-a5d8-45ca-9087-22fcc19c998c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:54.638167 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-d9040148-a5d8-45ca-9087-22fcc19c998c None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:26:54.639754 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-d9040148-a5d8-45ca-9087-22fcc19c998c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:26:54.668300 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:26:54.741307 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 32/65] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:26:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 17:26:54.879293 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08eec59e-b304-4813-b1fc-a56da3fa07bf None None] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:54.879943 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08eec59e-b304-4813-b1fc-a56da3fa07bf None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:54.880410 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08eec59e-b304-4813-b1fc-a56da3fa07bf None None] PATH_INFO: `/` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:54.882216 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 34/66] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:26:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:26:54.887237 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35187e67-9aa6-45fe-8fd7-518f29860b94 None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:54.887534 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35187e67-9aa6-45fe-8fd7-518f29860b94 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:54.887816 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35187e67-9aa6-45fe-8fd7-518f29860b94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:54.914847 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-35187e67-9aa6-45fe-8fd7-518f29860b94 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:26:54.916122 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-35187e67-9aa6-45fe-8fd7-518f29860b94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:26:54.945987 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08eec59e-b304-4813-b1fc-a56da3fa07bf None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:55.034412 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 33/67] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:26:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:26:55.053702 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c2ec34-082a-47c6-a656-48f1fe902cf9 None None] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:55.054218 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c2ec34-082a-47c6-a656-48f1fe902cf9 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:55.054646 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c2ec34-082a-47c6-a656-48f1fe902cf9 None None] PATH_INFO: `/` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:55.056155 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 34/68] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:26:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 17:26:55.062404 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7d0b3d-fa70-4e1c-8acb-1401936a1ef0 None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:55.062904 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7d0b3d-fa70-4e1c-8acb-1401936a1ef0 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:55.063340 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7d0b3d-fa70-4e1c-8acb-1401936a1ef0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:55.100578 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-de7d0b3d-fa70-4e1c-8acb-1401936a1ef0 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:26:55.100578 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-de7d0b3d-fa70-4e1c-8acb-1401936a1ef0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:26:55.111288 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:26:55.125453 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08eec59e-b304-4813-b1fc-a56da3fa07bf None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:55.184595 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 35/69] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:26:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 17:26:55.184595 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7d0b3d-fa70-4e1c-8acb-1401936a1ef0 None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:55.203583 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:26:55.269473 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a848a75c-1667-45b5-a2ff-715d6a3c11e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:55.270351 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a848a75c-1667-45b5-a2ff-715d6a3c11e4 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:55.270880 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a848a75c-1667-45b5-a2ff-715d6a3c11e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:55.271301 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a848a75c-1667-45b5-a2ff-715d6a3c11e4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:55.282687 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a848a75c-1667-45b5-a2ff-715d6a3c11e4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:26:55.288030 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application [None req-a848a75c-1667-45b5-a2ff-715d6a3c11e4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 17:26:55.288030 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:26:55.288030 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:26:55.288030 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:26:55.288030 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:26:55.288030 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:26:55.288030 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:26:55.288030 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:26:55.288030 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:26:55.288030 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:26:55.288030 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:26:55.288030 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:26:55.288030 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:26:55.288030 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:26:55.288030 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:26:55.288030 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:26:55.289088 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:26:55.289088 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:26:55.289088 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:26:55.289088 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:26:55.289088 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:26:55.289088 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:26:55.289088 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:26:55.289088 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:26:55.289088 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:26:55.289088 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:26:55.289088 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:26:55.289088 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application with Lock( Aug 24 17:26:55.289088 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:26:55.289088 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:26:55.289088 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:26:55.289088 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:26:55.290018 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:26:55.290018 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:26:55.290018 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:26:55.290018 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:26:55.290018 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:26:55.290018 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:26:55.290018 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:26:55.290018 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:26:55.290018 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:26:55.290018 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:26:55.290018 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 17:26:55.290018 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Aug 24 17:26:55.291787 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 35/70] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Aug 24 17:26:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 346 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 17:26:55.293085 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1aa582a-d11b-4a51-aba7-fcdc25745682 None None] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:55.293572 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1aa582a-d11b-4a51-aba7-fcdc25745682 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:55.294025 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1aa582a-d11b-4a51-aba7-fcdc25745682 None None] PATH_INFO: `/` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:55.295662 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 36/71] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:26:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:26:55.297779 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1aa582a-d11b-4a51-aba7-fcdc25745682 None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:55.301172 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7d0b3d-fa70-4e1c-8acb-1401936a1ef0 None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:55.338165 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c88e150-4911-46b3-bc58-1c7611edf5ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:55.339356 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c88e150-4911-46b3-bc58-1c7611edf5ba None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:55.339714 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c88e150-4911-46b3-bc58-1c7611edf5ba None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:55.339996 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c88e150-4911-46b3-bc58-1c7611edf5ba None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:55.351159 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c88e150-4911-46b3-bc58-1c7611edf5ba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:26:55.356815 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application [None req-0c88e150-4911-46b3-bc58-1c7611edf5ba None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:26:55.356815 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:26:55.356815 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:26:55.356815 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:26:55.356815 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:26:55.356815 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:26:55.356815 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:26:55.356815 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:26:55.356815 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:26:55.356815 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:26:55.356815 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:26:55.356815 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:26:55.356815 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:26:55.356815 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:26:55.356815 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:26:55.356815 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:26:55.357642 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:26:55.357642 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:26:55.357642 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:26:55.357642 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:26:55.357642 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:26:55.357642 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:26:55.357642 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:26:55.357642 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:26:55.357642 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:26:55.357642 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:26:55.357642 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:26:55.357642 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application with Lock( Aug 24 17:26:55.357642 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:26:55.357642 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:26:55.357642 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:26:55.357642 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:26:55.358384 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:26:55.358384 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:26:55.358384 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:26:55.358384 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:26:55.358384 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:26:55.358384 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:26:55.358384 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:26:55.358384 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:26:55.358384 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:26:55.358384 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:26:55.358384 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:26:55.358384 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Aug 24 17:26:55.358384 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 36/72] 10.222.0.45 () {56 vars in 1096 bytes} [Wed Aug 24 17:26:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:26:55.359657 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f642b0-1d0d-493c-8a1b-d44ddaf1fd5f None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:55.359948 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f642b0-1d0d-493c-8a1b-d44ddaf1fd5f None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:55.360222 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f642b0-1d0d-493c-8a1b-d44ddaf1fd5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:55.380409 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-73f642b0-1d0d-493c-8a1b-d44ddaf1fd5f None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:26:55.381635 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-73f642b0-1d0d-493c-8a1b-d44ddaf1fd5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:26:55.405097 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1aa582a-d11b-4a51-aba7-fcdc25745682 None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:55.439680 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfe2b5d-5c55-4d92-b9db-ea0a8a4dced4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:55.440283 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfe2b5d-5c55-4d92-b9db-ea0a8a4dced4 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:55.440587 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfe2b5d-5c55-4d92-b9db-ea0a8a4dced4 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:55.440866 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfe2b5d-5c55-4d92-b9db-ea0a8a4dced4 None admin] PATH_INFO: `/v3/roles` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:55.441530 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfe2b5d-5c55-4d92-b9db-ea0a8a4dced4 None admin] RBAC: Adding query filter params (name=member) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:26:55.441824 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfe2b5d-5c55-4d92-b9db-ea0a8a4dced4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:55.443219 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfe2b5d-5c55-4d92-b9db-ea0a8a4dced4 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:55.452969 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 37/73] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:26:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:26:55.459891 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 37/74] 10.222.0.45 () {56 vars in 1101 bytes} [Wed Aug 24 17:26:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:26:55.459891 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfe2b5d-5c55-4d92-b9db-ea0a8a4dced4 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:55.460545 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f642b0-1d0d-493c-8a1b-d44ddaf1fd5f None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:55.464600 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:26:55.472392 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:26:55.566010 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfe2b5d-5c55-4d92-b9db-ea0a8a4dced4 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:55.569498 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f642b0-1d0d-493c-8a1b-d44ddaf1fd5f None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:55.601841 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2242579c-926c-4abc-90ee-f1f4eee73b1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:55.602718 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2242579c-926c-4abc-90ee-f1f4eee73b1d None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:55.603004 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2242579c-926c-4abc-90ee-f1f4eee73b1d None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:55.603280 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2242579c-926c-4abc-90ee-f1f4eee73b1d None admin] PATH_INFO: `/v3/roles` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:55.604023 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2242579c-926c-4abc-90ee-f1f4eee73b1d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:26:55.604329 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2242579c-926c-4abc-90ee-f1f4eee73b1d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:55.605407 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69619c3e-914e-4b1c-b687-a627f09fd0c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:55.605824 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2242579c-926c-4abc-90ee-f1f4eee73b1d None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:55.606045 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69619c3e-914e-4b1c-b687-a627f09fd0c7 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:55.606324 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69619c3e-914e-4b1c-b687-a627f09fd0c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:55.606597 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69619c3e-914e-4b1c-b687-a627f09fd0c7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:55.618576 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 38/75] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Aug 24 17:26:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:26:55.618576 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2242579c-926c-4abc-90ee-f1f4eee73b1d None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:55.622660 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69619c3e-914e-4b1c-b687-a627f09fd0c7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:26:55.628228 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application [None req-69619c3e-914e-4b1c-b687-a627f09fd0c7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 17:26:55.628228 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:26:55.628228 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:26:55.628228 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:26:55.628228 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:26:55.628228 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:26:55.628228 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:26:55.628228 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:26:55.628228 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:26:55.628228 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:26:55.628228 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:26:55.628228 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:26:55.628228 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:26:55.628228 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:26:55.628228 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:26:55.628228 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:26:55.629111 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:26:55.629111 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:26:55.629111 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:26:55.629111 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:26:55.629111 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:26:55.629111 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:26:55.629111 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:26:55.629111 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:26:55.629111 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:26:55.629111 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:26:55.629111 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:26:55.629111 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application with Lock( Aug 24 17:26:55.629111 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:26:55.629111 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:26:55.629111 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:26:55.629111 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:26:55.629907 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:26:55.629907 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:26:55.629907 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:26:55.629907 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:26:55.629907 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:26:55.629907 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:26:55.629907 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:26:55.629907 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:26:55.629907 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:26:55.629907 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:26:55.629907 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 17:26:55.629907 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Aug 24 17:26:55.629907 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 38/76] 10.222.0.45 () {56 vars in 1114 bytes} [Wed Aug 24 17:26:55 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:26:55.630533 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69619c3e-914e-4b1c-b687-a627f09fd0c7 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:55.692795 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:26:55.703178 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:26:55.732187 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2242579c-926c-4abc-90ee-f1f4eee73b1d None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:55.741410 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69619c3e-914e-4b1c-b687-a627f09fd0c7 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:55.766519 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28084cfe-acfe-42c9-b803-7c0c3056b4d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:55.767152 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28084cfe-acfe-42c9-b803-7c0c3056b4d1 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:55.767433 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28084cfe-acfe-42c9-b803-7c0c3056b4d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:55.767765 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28084cfe-acfe-42c9-b803-7c0c3056b4d1 None admin] PATH_INFO: `/v3/users/1f7a270ae41e45eab15c82f41826ca94` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:55.775494 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4837abc9-d4ca-4d48-a849-aa4e01070b91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:55.776159 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4837abc9-d4ca-4d48-a849-aa4e01070b91 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:55.776445 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4837abc9-d4ca-4d48-a849-aa4e01070b91 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:55.776755 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4837abc9-d4ca-4d48-a849-aa4e01070b91 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:55.777903 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4837abc9-d4ca-4d48-a849-aa4e01070b91 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:55.779365 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4837abc9-d4ca-4d48-a849-aa4e01070b91 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:55.780984 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 39/77] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 17:26:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:26:55.781749 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4837abc9-d4ca-4d48-a849-aa4e01070b91 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:55.789606 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28084cfe-acfe-42c9-b803-7c0c3056b4d1 None admin] RBAC: Authorizing `identity:get_user(user_id=1f7a270ae41e45eab15c82f41826ca94)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:55.791108 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28084cfe-acfe-42c9-b803-7c0c3056b4d1 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:55.792829 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 39/78] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Aug 24 17:26:55 2022] GET /identity/v3/users/1f7a270ae41e45eab15c82f41826ca94 => generated 275 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:26:55.793600 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28084cfe-acfe-42c9-b803-7c0c3056b4d1 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:55.885735 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4837abc9-d4ca-4d48-a849-aa4e01070b91 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:55.894233 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28084cfe-acfe-42c9-b803-7c0c3056b4d1 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:55.916747 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:26:55.918657 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95432b8b-95f2-4035-b736-fe25cc2be7a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:55.919258 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95432b8b-95f2-4035-b736-fe25cc2be7a0 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:55.919544 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95432b8b-95f2-4035-b736-fe25cc2be7a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:55.919825 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95432b8b-95f2-4035-b736-fe25cc2be7a0 None admin] PATH_INFO: `/v3/users/cc2669e1aa494dbcb7164d2e5e46378c` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:55.921013 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95432b8b-95f2-4035-b736-fe25cc2be7a0 None admin] RBAC: Authorizing `identity:get_user(user_id=cc2669e1aa494dbcb7164d2e5e46378c)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:55.922458 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95432b8b-95f2-4035-b736-fe25cc2be7a0 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:55.924034 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 40/79] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Aug 24 17:26:55 2022] GET /identity/v3/users/cc2669e1aa494dbcb7164d2e5e46378c => generated 247 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:26:55.927441 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d545e70e-b190-4542-b108-4757a8740b44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:55.928045 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d545e70e-b190-4542-b108-4757a8740b44 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:55.928327 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d545e70e-b190-4542-b108-4757a8740b44 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:55.928419 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95432b8b-95f2-4035-b736-fe25cc2be7a0 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:55.928665 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d545e70e-b190-4542-b108-4757a8740b44 None admin] PATH_INFO: `/v3/roles` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:55.929278 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d545e70e-b190-4542-b108-4757a8740b44 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:26:55.929580 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d545e70e-b190-4542-b108-4757a8740b44 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:55.930946 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d545e70e-b190-4542-b108-4757a8740b44 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:55.931149 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:26:55.940157 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 40/80] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Aug 24 17:26:55 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:26:55.940775 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d545e70e-b190-4542-b108-4757a8740b44 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:56.028314 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95432b8b-95f2-4035-b736-fe25cc2be7a0 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:56.039438 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d545e70e-b190-4542-b108-4757a8740b44 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:56.061359 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c268ba-10a7-474b-9c4f-19eae60ad15c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:56.061990 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c268ba-10a7-474b-9c4f-19eae60ad15c None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:56.062270 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c268ba-10a7-474b-9c4f-19eae60ad15c None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:56.062552 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c268ba-10a7-474b-9c4f-19eae60ad15c None admin] PATH_INFO: `/v3/roles/member` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:56.072316 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c268ba-10a7-474b-9c4f-19eae60ad15c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:26:56.073161 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ac7b12-686d-47af-b507-38c084503c0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:56.073773 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ac7b12-686d-47af-b507-38c084503c0d None admin] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:56.074060 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ac7b12-686d-47af-b507-38c084503c0d None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:56.074342 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ac7b12-686d-47af-b507-38c084503c0d None admin] PATH_INFO: `/v3/users` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:56.074988 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ac7b12-686d-47af-b507-38c084503c0d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:56.076448 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ac7b12-686d-47af-b507-38c084503c0d None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:56.077551 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application [None req-40c268ba-10a7-474b-9c4f-19eae60ad15c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 17:26:56.077551 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:26:56.077551 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:26:56.077551 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:26:56.077551 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:26:56.077551 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:26:56.077551 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:26:56.077551 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:26:56.077551 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:26:56.077551 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:26:56.077551 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:26:56.077551 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:26:56.077551 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:26:56.077551 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:26:56.077551 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:26:56.077551 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:26:56.078509 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:26:56.078509 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:26:56.078509 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:26:56.078509 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:26:56.078509 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:26:56.078509 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:26:56.078509 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:26:56.078509 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:26:56.078509 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:26:56.078509 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:26:56.078509 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:26:56.078509 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application with Lock( Aug 24 17:26:56.078509 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:26:56.078509 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:26:56.078509 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:26:56.078509 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:26:56.079238 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:26:56.079238 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:26:56.079238 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:26:56.079238 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:26:56.079238 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:26:56.079238 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:26:56.079238 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:26:56.079238 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:26:56.079238 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:26:56.079238 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:26:56.079238 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 17:26:56.079238 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Aug 24 17:26:56.079238 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 41/81] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Aug 24 17:26:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:26:56.080733 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c268ba-10a7-474b-9c4f-19eae60ad15c None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:56.096909 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 41/82] 10.222.0.45 () {60 vars in 1132 bytes} [Wed Aug 24 17:26:55 2022] POST /identity/v3/users => generated 289 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:26:56.097737 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ac7b12-686d-47af-b507-38c084503c0d None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:56.148326 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:26:56.157220 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:26:56.202408 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c268ba-10a7-474b-9c4f-19eae60ad15c None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:56.219077 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ac7b12-686d-47af-b507-38c084503c0d None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:56.240583 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e0876b-7f8a-46d9-876c-2a22f411e05c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:56.241345 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e0876b-7f8a-46d9-876c-2a22f411e05c None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:56.241731 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e0876b-7f8a-46d9-876c-2a22f411e05c None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:56.242102 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e0876b-7f8a-46d9-876c-2a22f411e05c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:56.266173 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5bd217-446d-49c5-b55f-475cef73e697 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:56.266804 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5bd217-446d-49c5-b55f-475cef73e697 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:56.267090 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5bd217-446d-49c5-b55f-475cef73e697 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:56.267733 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5bd217-446d-49c5-b55f-475cef73e697 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:56.327782 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e0876b-7f8a-46d9-876c-2a22f411e05c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:56.330054 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e0876b-7f8a-46d9-876c-2a22f411e05c None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:56.345292 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5bd217-446d-49c5-b55f-475cef73e697 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:56.346645 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5bd217-446d-49c5-b55f-475cef73e697 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:56.406761 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:26:56.407907 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:26:56.453029 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 42/83] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Aug 24 17:26:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 17:26:56.453889 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e0876b-7f8a-46d9-876c-2a22f411e05c None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:56.488205 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 42/84] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Aug 24 17:26:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 17:26:56.489387 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5bd217-446d-49c5-b55f-475cef73e697 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:56.566464 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e0876b-7f8a-46d9-876c-2a22f411e05c None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:56.603233 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5bd217-446d-49c5-b55f-475cef73e697 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:56.603736 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee5bf58-aa35-403e-9442-00d8d29e569f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:56.604589 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee5bf58-aa35-403e-9442-00d8d29e569f None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:56.608586 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee5bf58-aa35-403e-9442-00d8d29e569f None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:56.608586 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee5bf58-aa35-403e-9442-00d8d29e569f None admin] PATH_INFO: `/v3/users/1f7a270ae41e45eab15c82f41826ca94` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:56.608586 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee5bf58-aa35-403e-9442-00d8d29e569f None admin] RBAC: Authorizing `identity:get_user(user_id=1f7a270ae41e45eab15c82f41826ca94)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:56.608586 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee5bf58-aa35-403e-9442-00d8d29e569f None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:56.609511 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 43/85] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Aug 24 17:26:56 2022] GET /identity/v3/users/1f7a270ae41e45eab15c82f41826ca94 => generated 275 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:26:56.610337 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee5bf58-aa35-403e-9442-00d8d29e569f None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:56.641585 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:26:56.649551 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cde6d3b-0c8f-4905-b85f-0048c8a5b64c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:56.650432 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cde6d3b-0c8f-4905-b85f-0048c8a5b64c None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:56.651468 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cde6d3b-0c8f-4905-b85f-0048c8a5b64c None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:56.651890 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cde6d3b-0c8f-4905-b85f-0048c8a5b64c None admin] PATH_INFO: `/v3/roles` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:56.652738 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cde6d3b-0c8f-4905-b85f-0048c8a5b64c None admin] RBAC: Adding query filter params (name=member) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:26:56.653331 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cde6d3b-0c8f-4905-b85f-0048c8a5b64c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:56.654893 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cde6d3b-0c8f-4905-b85f-0048c8a5b64c None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:56.664615 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 43/86] 10.222.0.45 () {56 vars in 1101 bytes} [Wed Aug 24 17:26:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:26:56.665699 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cde6d3b-0c8f-4905-b85f-0048c8a5b64c None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:56.669696 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:26:56.716650 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee5bf58-aa35-403e-9442-00d8d29e569f None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:56.751556 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a05c753-e016-4f9f-9b6f-612d9c2f764f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:56.752357 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a05c753-e016-4f9f-9b6f-612d9c2f764f None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:56.752682 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a05c753-e016-4f9f-9b6f-612d9c2f764f None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:56.752964 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a05c753-e016-4f9f-9b6f-612d9c2f764f None admin] PATH_INFO: `/v3/projects/cc1182fc1781499884cebfaacd1ef5a0` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:56.754476 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a05c753-e016-4f9f-9b6f-612d9c2f764f None admin] RBAC: Authorizing `identity:get_project(project_id=cc1182fc1781499884cebfaacd1ef5a0)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:56.756214 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a05c753-e016-4f9f-9b6f-612d9c2f764f None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:56.758394 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 44/87] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Aug 24 17:26:56 2022] GET /identity/v3/projects/cc1182fc1781499884cebfaacd1ef5a0 => generated 298 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:26:56.759165 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a05c753-e016-4f9f-9b6f-612d9c2f764f None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:56.773477 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cde6d3b-0c8f-4905-b85f-0048c8a5b64c None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:56.811734 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c774bcc8-e6d2-411f-a2b7-82355c917977 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:56.812706 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c774bcc8-e6d2-411f-a2b7-82355c917977 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:56.813195 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c774bcc8-e6d2-411f-a2b7-82355c917977 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:56.813687 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c774bcc8-e6d2-411f-a2b7-82355c917977 None admin] PATH_INFO: `/v3/projects/cc1182fc1781499884cebfaacd1ef5a0` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:56.815069 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c774bcc8-e6d2-411f-a2b7-82355c917977 None admin] RBAC: Authorizing `identity:get_project(project_id=cc1182fc1781499884cebfaacd1ef5a0)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:56.816788 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c774bcc8-e6d2-411f-a2b7-82355c917977 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:56.818920 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 44/88] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Aug 24 17:26:56 2022] GET /identity/v3/projects/cc1182fc1781499884cebfaacd1ef5a0 => generated 298 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:26:56.819890 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c774bcc8-e6d2-411f-a2b7-82355c917977 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:56.884915 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:26:56.892497 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a05c753-e016-4f9f-9b6f-612d9c2f764f None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:56.927976 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:26:56.929161 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dffe99c-c2ad-4364-b6e3-251513749121 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:56.929785 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dffe99c-c2ad-4364-b6e3-251513749121 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:56.930067 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dffe99c-c2ad-4364-b6e3-251513749121 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:56.930463 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dffe99c-c2ad-4364-b6e3-251513749121 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:56.956803 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c774bcc8-e6d2-411f-a2b7-82355c917977 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:57.003446 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df02c1e-119a-4945-92c0-39b4769215d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:57.004714 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df02c1e-119a-4945-92c0-39b4769215d5 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:57.005822 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df02c1e-119a-4945-92c0-39b4769215d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:57.006429 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df02c1e-119a-4945-92c0-39b4769215d5 None admin] PATH_INFO: `/v3/users/1f7a270ae41e45eab15c82f41826ca94` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:57.007601 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dffe99c-c2ad-4364-b6e3-251513749121 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:57.008114 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df02c1e-119a-4945-92c0-39b4769215d5 None admin] RBAC: Authorizing `identity:get_user(user_id=1f7a270ae41e45eab15c82f41826ca94)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:57.009113 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dffe99c-c2ad-4364-b6e3-251513749121 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:57.009994 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df02c1e-119a-4945-92c0-39b4769215d5 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:57.013305 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 45/89] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Aug 24 17:26:56 2022] GET /identity/v3/users/1f7a270ae41e45eab15c82f41826ca94 => generated 275 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:26:57.014448 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df02c1e-119a-4945-92c0-39b4769215d5 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:57.109811 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:26:57.115375 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 45/90] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Aug 24 17:26:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 17:26:57.116571 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dffe99c-c2ad-4364-b6e3-251513749121 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:57.149909 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df02c1e-119a-4945-92c0-39b4769215d5 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:57.185032 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9476f513-429f-424a-923a-6ea708a2dcba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:57.185747 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9476f513-429f-424a-923a-6ea708a2dcba None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:57.186132 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9476f513-429f-424a-923a-6ea708a2dcba None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:57.186503 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9476f513-429f-424a-923a-6ea708a2dcba None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:57.187327 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9476f513-429f-424a-923a-6ea708a2dcba None admin] RBAC: Adding query filter params (role.id=02476a5b82eb4b188c18a5d706a3ef68, scope.project.id=cc1182fc1781499884cebfaacd1ef5a0, user.id=1f7a270ae41e45eab15c82f41826ca94) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:26:57.187739 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9476f513-429f-424a-923a-6ea708a2dcba None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:57.189580 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9476f513-429f-424a-923a-6ea708a2dcba None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:57.198943 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 46/91] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Aug 24 17:26:57 2022] GET /identity/v3/role_assignments?user.id=1f7a270ae41e45eab15c82f41826ca94&scope.project.id=cc1182fc1781499884cebfaacd1ef5a0&role.id=02476a5b82eb4b188c18a5d706a3ef68 => generated 259 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:26:57.205856 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9476f513-429f-424a-923a-6ea708a2dcba None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:57.224973 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:26:57.241960 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dffe99c-c2ad-4364-b6e3-251513749121 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:57.284648 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116e69c1-9cd5-4ac0-ae8c-96b5e6d383c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:57.285512 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116e69c1-9cd5-4ac0-ae8c-96b5e6d383c1 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:57.285939 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116e69c1-9cd5-4ac0-ae8c-96b5e6d383c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:57.286356 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116e69c1-9cd5-4ac0-ae8c-96b5e6d383c1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:57.287238 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116e69c1-9cd5-4ac0-ae8c-96b5e6d383c1 None admin] RBAC: Adding query filter params (role.id=e411ad9eb96e4e8d85b59b5b700ada7b, scope.project.id=cc1182fc1781499884cebfaacd1ef5a0, user.id=cc2669e1aa494dbcb7164d2e5e46378c) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:26:57.287717 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116e69c1-9cd5-4ac0-ae8c-96b5e6d383c1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:57.289342 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116e69c1-9cd5-4ac0-ae8c-96b5e6d383c1 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:57.296693 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 46/92] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Aug 24 17:26:57 2022] GET /identity/v3/role_assignments?user.id=cc2669e1aa494dbcb7164d2e5e46378c&scope.project.id=cc1182fc1781499884cebfaacd1ef5a0&role.id=e411ad9eb96e4e8d85b59b5b700ada7b => generated 259 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:26:57.297825 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116e69c1-9cd5-4ac0-ae8c-96b5e6d383c1 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:57.399848 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9476f513-429f-424a-923a-6ea708a2dcba None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:57.422690 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:26:57.459731 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868c3895-5b85-4fae-b6de-746c99988ce0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:57.459731 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868c3895-5b85-4fae-b6de-746c99988ce0 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:57.459731 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868c3895-5b85-4fae-b6de-746c99988ce0 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:57.459731 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868c3895-5b85-4fae-b6de-746c99988ce0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:57.527402 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116e69c1-9cd5-4ac0-ae8c-96b5e6d383c1 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:57.548814 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868c3895-5b85-4fae-b6de-746c99988ce0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:57.550587 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868c3895-5b85-4fae-b6de-746c99988ce0 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:57.576999 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:26:57.615275 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da90b3f-359a-498a-8d48-4b78a0c93a6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:57.616586 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da90b3f-359a-498a-8d48-4b78a0c93a6c None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:57.617069 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da90b3f-359a-498a-8d48-4b78a0c93a6c None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:57.617520 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da90b3f-359a-498a-8d48-4b78a0c93a6c None admin] PATH_INFO: `/v3/projects/cc1182fc1781499884cebfaacd1ef5a0` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:57.618842 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da90b3f-359a-498a-8d48-4b78a0c93a6c None admin] RBAC: Authorizing `identity:get_project(project_id=cc1182fc1781499884cebfaacd1ef5a0)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:57.620522 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da90b3f-359a-498a-8d48-4b78a0c93a6c None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:57.622419 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 47/93] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Aug 24 17:26:57 2022] GET /identity/v3/projects/cc1182fc1781499884cebfaacd1ef5a0 => generated 298 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:26:57.631244 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da90b3f-359a-498a-8d48-4b78a0c93a6c None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:57.718410 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 47/94] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Aug 24 17:26:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 519 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 17:26:57.725649 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868c3895-5b85-4fae-b6de-746c99988ce0 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:57.845364 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868c3895-5b85-4fae-b6de-746c99988ce0 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:57.868939 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:26:57.883474 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da90b3f-359a-498a-8d48-4b78a0c93a6c None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:57.888047 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c205b37-6d12-440b-9a45-4b0817745d68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:57.888731 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c205b37-6d12-440b-9a45-4b0817745d68 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:57.889025 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c205b37-6d12-440b-9a45-4b0817745d68 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:57.889307 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c205b37-6d12-440b-9a45-4b0817745d68 None admin] PATH_INFO: `/v3/projects/e615e00f222d4a298a1f7ceea74eb618` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:57.890483 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c205b37-6d12-440b-9a45-4b0817745d68 None admin] RBAC: Authorizing `identity:get_project(project_id=e615e00f222d4a298a1f7ceea74eb618)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:57.892061 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c205b37-6d12-440b-9a45-4b0817745d68 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:57.893967 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 48/95] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Aug 24 17:26:57 2022] GET /identity/v3/projects/e615e00f222d4a298a1f7ceea74eb618 => generated 312 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:26:57.899514 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c205b37-6d12-440b-9a45-4b0817745d68 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:57.929918 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:26:57.949841 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6395184-2b94-4ce1-8aa9-f2eb29a3f468 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:57.950470 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6395184-2b94-4ce1-8aa9-f2eb29a3f468 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:57.950752 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6395184-2b94-4ce1-8aa9-f2eb29a3f468 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:57.951028 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6395184-2b94-4ce1-8aa9-f2eb29a3f468 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:57.951769 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6395184-2b94-4ce1-8aa9-f2eb29a3f468 None admin] RBAC: Adding query filter params (role.id=148f135e4a134697a032be04bc9faa4c, scope.project.id=cc1182fc1781499884cebfaacd1ef5a0, user.id=1f7a270ae41e45eab15c82f41826ca94) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:26:57.952077 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6395184-2b94-4ce1-8aa9-f2eb29a3f468 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:57.953577 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6395184-2b94-4ce1-8aa9-f2eb29a3f468 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:57.967740 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 48/96] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Aug 24 17:26:57 2022] GET /identity/v3/role_assignments?user.id=1f7a270ae41e45eab15c82f41826ca94&scope.project.id=cc1182fc1781499884cebfaacd1ef5a0&role.id=148f135e4a134697a032be04bc9faa4c => generated 259 bytes in 337 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:26:58.008484 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c205b37-6d12-440b-9a45-4b0817745d68 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:58.049972 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59debb96-96ab-41d6-89fa-654039215e3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:58.050605 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59debb96-96ab-41d6-89fa-654039215e3e None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:58.050888 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59debb96-96ab-41d6-89fa-654039215e3e None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:58.051164 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59debb96-96ab-41d6-89fa-654039215e3e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:58.051908 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59debb96-96ab-41d6-89fa-654039215e3e None admin] RBAC: Adding query filter params (role.id=02476a5b82eb4b188c18a5d706a3ef68, scope.project.id=e615e00f222d4a298a1f7ceea74eb618, user.id=1f7a270ae41e45eab15c82f41826ca94) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:26:58.052219 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59debb96-96ab-41d6-89fa-654039215e3e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:58.053719 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59debb96-96ab-41d6-89fa-654039215e3e None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:58.060508 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 49/97] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Aug 24 17:26:57 2022] GET /identity/v3/role_assignments?user.id=1f7a270ae41e45eab15c82f41826ca94&scope.project.id=e615e00f222d4a298a1f7ceea74eb618&role.id=02476a5b82eb4b188c18a5d706a3ef68 => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:26:58.562425 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df13ee91-a5f8-4fff-a5ec-d146bcd91589 None None] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:58.562771 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df13ee91-a5f8-4fff-a5ec-d146bcd91589 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:58.562982 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df13ee91-a5f8-4fff-a5ec-d146bcd91589 None None] PATH_INFO: `/` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:58.564337 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 49/98] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:26:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:26:58.582209 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d63f1f-0265-40b5-bdf8-4c68aa7ecd66 None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:58.582503 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d63f1f-0265-40b5-bdf8-4c68aa7ecd66 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:58.582774 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d63f1f-0265-40b5-bdf8-4c68aa7ecd66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:58.606630 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-32d63f1f-0265-40b5-bdf8-4c68aa7ecd66 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:26:58.607836 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-32d63f1f-0265-40b5-bdf8-4c68aa7ecd66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:26:58.642959 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:26:58.647157 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec0f839-fc1a-46e1-9071-b1f7cf73e01f None None] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:58.647634 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec0f839-fc1a-46e1-9071-b1f7cf73e01f None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:58.647708 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec0f839-fc1a-46e1-9071-b1f7cf73e01f None None] PATH_INFO: `/` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:58.649062 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 50/99] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:26:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 17:26:58.661500 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fa9552-1ecd-4215-9a21-bf2b3123b778 None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:58.661774 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fa9552-1ecd-4215-9a21-bf2b3123b778 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:58.662055 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fa9552-1ecd-4215-9a21-bf2b3123b778 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:58.693040 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-29fa9552-1ecd-4215-9a21-bf2b3123b778 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:26:58.694259 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-29fa9552-1ecd-4215-9a21-bf2b3123b778 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:26:58.717800 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 50/100] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:26:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:26:58.823248 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 51/101] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:26:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 17:26:59.050540 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d63f1f-0265-40b5-bdf8-4c68aa7ecd66 None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:59.177993 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d63f1f-0265-40b5-bdf8-4c68aa7ecd66 None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:59.212535 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22625f5c-6765-41f0-a6f1-dfec0f89a038 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:59.213250 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22625f5c-6765-41f0-a6f1-dfec0f89a038 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:59.214023 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22625f5c-6765-41f0-a6f1-dfec0f89a038 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:59.214298 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22625f5c-6765-41f0-a6f1-dfec0f89a038 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:59.224516 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22625f5c-6765-41f0-a6f1-dfec0f89a038 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:26:59.229640 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application [None req-22625f5c-6765-41f0-a6f1-dfec0f89a038 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 17:26:59.229640 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:26:59.229640 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:26:59.229640 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:26:59.229640 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:26:59.229640 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:26:59.229640 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:26:59.229640 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:26:59.229640 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:26:59.229640 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:26:59.229640 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:26:59.229640 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:26:59.229640 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:26:59.229640 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:26:59.229640 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:26:59.229640 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:26:59.230471 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:26:59.230471 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:26:59.230471 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:26:59.230471 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:26:59.230471 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:26:59.230471 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:26:59.230471 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:26:59.230471 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:26:59.230471 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:26:59.230471 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:26:59.230471 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:26:59.230471 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application with Lock( Aug 24 17:26:59.230471 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:26:59.230471 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:26:59.230471 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:26:59.230471 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:26:59.231626 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:26:59.231626 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:26:59.231626 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:26:59.231626 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:26:59.231626 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:26:59.231626 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:26:59.231626 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:26:59.231626 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:26:59.231626 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:26:59.231626 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:26:59.231626 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 17:26:59.231626 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Aug 24 17:26:59.231626 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 51/102] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Aug 24 17:26:59 2022] GET /identity/v3/roles/reader => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:26:59.236484 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fa9552-1ecd-4215-9a21-bf2b3123b778 None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:59.336858 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fa9552-1ecd-4215-9a21-bf2b3123b778 None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:59.353249 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22625f5c-6765-41f0-a6f1-dfec0f89a038 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:59.381013 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:26:59.383671 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615128c4-4a63-4969-a438-693c33ab805c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:59.384303 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615128c4-4a63-4969-a438-693c33ab805c None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:59.384601 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615128c4-4a63-4969-a438-693c33ab805c None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:59.384886 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615128c4-4a63-4969-a438-693c33ab805c None admin] PATH_INFO: `/v3/roles` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:59.385683 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615128c4-4a63-4969-a438-693c33ab805c None admin] RBAC: Adding query filter params (name=reader) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:26:59.385989 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615128c4-4a63-4969-a438-693c33ab805c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:59.387382 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615128c4-4a63-4969-a438-693c33ab805c None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:59.405004 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 52/103] 10.222.0.45 () {56 vars in 1101 bytes} [Wed Aug 24 17:26:59 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:26:59.409029 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:26:59.413509 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615128c4-4a63-4969-a438-693c33ab805c None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:59.493037 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22625f5c-6765-41f0-a6f1-dfec0f89a038 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:59.547276 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43069bd-5469-43ab-90cc-a79926684815 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:59.547947 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43069bd-5469-43ab-90cc-a79926684815 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:59.548231 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43069bd-5469-43ab-90cc-a79926684815 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:59.548509 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43069bd-5469-43ab-90cc-a79926684815 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:59.551989 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43069bd-5469-43ab-90cc-a79926684815 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:59.553559 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43069bd-5469-43ab-90cc-a79926684815 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:59.555285 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 52/104] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 17:26:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:26:59.561083 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43069bd-5469-43ab-90cc-a79926684815 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:59.711414 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43069bd-5469-43ab-90cc-a79926684815 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:59.722577 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615128c4-4a63-4969-a438-693c33ab805c None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:59.745879 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:26:59.750155 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9447bd2-602e-45d8-90b1-466a11ed771c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:59.750775 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9447bd2-602e-45d8-90b1-466a11ed771c None admin] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:59.751058 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9447bd2-602e-45d8-90b1-466a11ed771c None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:59.751333 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9447bd2-602e-45d8-90b1-466a11ed771c None admin] PATH_INFO: `/v3/projects` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:59.751981 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9447bd2-602e-45d8-90b1-466a11ed771c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:59.753493 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9447bd2-602e-45d8-90b1-466a11ed771c None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:59.806147 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 53/105] 10.222.0.45 () {60 vars in 1141 bytes} [Wed Aug 24 17:26:59 2022] POST /identity/v3/projects => generated 302 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:26:59.808433 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d1bb2e-8a48-4d25-9ea7-3a29c013db80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:59.809238 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d1bb2e-8a48-4d25-9ea7-3a29c013db80 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:59.809785 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d1bb2e-8a48-4d25-9ea7-3a29c013db80 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:59.810268 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d1bb2e-8a48-4d25-9ea7-3a29c013db80 None admin] PATH_INFO: `/v3/users/9bac15d3c1cc40f9bbfa687de74340b4` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:59.826923 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d1bb2e-8a48-4d25-9ea7-3a29c013db80 None admin] RBAC: Authorizing `identity:get_user(user_id=9bac15d3c1cc40f9bbfa687de74340b4)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:59.828791 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d1bb2e-8a48-4d25-9ea7-3a29c013db80 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:59.830802 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 53/106] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Aug 24 17:26:59 2022] GET /identity/v3/users/9bac15d3c1cc40f9bbfa687de74340b4 => generated 289 bytes in 418 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:26:59.841707 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9447bd2-602e-45d8-90b1-466a11ed771c None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:59.915772 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2f451b-27c3-4def-bfb1-1001aa4532ff None None] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:59.916947 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2f451b-27c3-4def-bfb1-1001aa4532ff None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:59.917437 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2f451b-27c3-4def-bfb1-1001aa4532ff None None] PATH_INFO: `/` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:59.919311 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 54/107] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:26:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:26:59.928896 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fc2bdd-e817-4ffd-8ac7-15b863a983c6 None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:59.929491 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fc2bdd-e817-4ffd-8ac7-15b863a983c6 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:59.929969 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fc2bdd-e817-4ffd-8ac7-15b863a983c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:59.963097 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-b6fc2bdd-e817-4ffd-8ac7-15b863a983c6 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:26:59.964719 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-b6fc2bdd-e817-4ffd-8ac7-15b863a983c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:26:59.985602 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9447bd2-602e-45d8-90b1-466a11ed771c None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:00.027955 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d80adf-75a0-43c7-ad63-bdafa86a60c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:00.029095 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d80adf-75a0-43c7-ad63-bdafa86a60c4 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:00.029526 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d80adf-75a0-43c7-ad63-bdafa86a60c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:00.029946 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d80adf-75a0-43c7-ad63-bdafa86a60c4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:00.056993 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:00.065898 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:00.131696 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 55/108] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:26:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:27:00.163985 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d80adf-75a0-43c7-ad63-bdafa86a60c4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:00.165571 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d80adf-75a0-43c7-ad63-bdafa86a60c4 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:00.283545 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 54/109] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Aug 24 17:26:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 17:27:00.289107 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fc2bdd-e817-4ffd-8ac7-15b863a983c6 None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:00.418562 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fc2bdd-e817-4ffd-8ac7-15b863a983c6 None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:00.452718 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f53011d-3a28-49f1-b799-34a3d9e11686 None None] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:00.453371 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f53011d-3a28-49f1-b799-34a3d9e11686 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:00.453850 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f53011d-3a28-49f1-b799-34a3d9e11686 None None] PATH_INFO: `/` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:00.455616 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 55/110] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:27:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:27:00.460861 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9e07cd-aaea-476f-912b-f6522ea834ab None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:00.461365 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9e07cd-aaea-476f-912b-f6522ea834ab None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:00.461939 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9e07cd-aaea-476f-912b-f6522ea834ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:00.462485 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c58c83-8a72-4642-9ea2-653a0b3129e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:00.463122 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c58c83-8a72-4642-9ea2-653a0b3129e7 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:00.463463 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c58c83-8a72-4642-9ea2-653a0b3129e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:00.463740 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c58c83-8a72-4642-9ea2-653a0b3129e7 None admin] PATH_INFO: `/v3/projects/cc1182fc1781499884cebfaacd1ef5a0` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:00.464928 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c58c83-8a72-4642-9ea2-653a0b3129e7 None admin] RBAC: Authorizing `identity:get_project(project_id=cc1182fc1781499884cebfaacd1ef5a0)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:00.466342 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c58c83-8a72-4642-9ea2-653a0b3129e7 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:00.467972 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 56/111] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Aug 24 17:27:00 2022] GET /identity/v3/projects/cc1182fc1781499884cebfaacd1ef5a0 => generated 298 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:27:00.473775 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c58c83-8a72-4642-9ea2-653a0b3129e7 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:00.493800 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-1e9e07cd-aaea-476f-912b-f6522ea834ab None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:27:00.496042 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-1e9e07cd-aaea-476f-912b-f6522ea834ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:27:00.517624 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:00.552956 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:00.601372 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c58c83-8a72-4642-9ea2-653a0b3129e7 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:00.603396 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 56/112] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:27:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 17:27:00.604449 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9e07cd-aaea-476f-912b-f6522ea834ab None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:00.648578 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b223d4d8-876a-40af-bb8b-1390d5ae8b6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:00.648578 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b223d4d8-876a-40af-bb8b-1390d5ae8b6b None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:00.648578 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b223d4d8-876a-40af-bb8b-1390d5ae8b6b None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:00.648578 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b223d4d8-876a-40af-bb8b-1390d5ae8b6b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:00.648972 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b223d4d8-876a-40af-bb8b-1390d5ae8b6b None admin] RBAC: Adding query filter params (role.id=645db6d227604357b1949889630755ff, scope.project.id=cc1182fc1781499884cebfaacd1ef5a0, user.id=9bac15d3c1cc40f9bbfa687de74340b4) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:27:00.649219 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b223d4d8-876a-40af-bb8b-1390d5ae8b6b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:00.651401 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b223d4d8-876a-40af-bb8b-1390d5ae8b6b None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:00.661489 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 57/113] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Aug 24 17:27:00 2022] GET /identity/v3/role_assignments?user.id=9bac15d3c1cc40f9bbfa687de74340b4&scope.project.id=cc1182fc1781499884cebfaacd1ef5a0&role.id=645db6d227604357b1949889630755ff => generated 259 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:27:00.745959 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9e07cd-aaea-476f-912b-f6522ea834ab None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:00.789764 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb46bc75-764a-4c67-b9bc-1755277b52ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:00.790754 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb46bc75-764a-4c67-b9bc-1755277b52ab None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:00.791299 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb46bc75-764a-4c67-b9bc-1755277b52ab None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:00.791802 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb46bc75-764a-4c67-b9bc-1755277b52ab None admin] PATH_INFO: `/v3/roles/member` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:00.803869 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb46bc75-764a-4c67-b9bc-1755277b52ab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:27:00.809689 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application [None req-bb46bc75-764a-4c67-b9bc-1755277b52ab None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 17:27:00.809689 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:27:00.809689 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:27:00.809689 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:27:00.809689 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:27:00.809689 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:27:00.809689 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:27:00.809689 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:27:00.809689 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:27:00.809689 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:27:00.809689 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:27:00.809689 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:27:00.809689 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:27:00.809689 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:27:00.809689 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:27:00.809689 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:27:00.810581 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:27:00.810581 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:27:00.810581 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:27:00.810581 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:27:00.810581 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:27:00.810581 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:27:00.810581 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:27:00.810581 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:27:00.810581 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:27:00.810581 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:27:00.810581 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:27:00.810581 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application with Lock( Aug 24 17:27:00.810581 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:27:00.810581 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:27:00.810581 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:27:00.810581 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:27:00.811317 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:27:00.811317 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:27:00.811317 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:27:00.811317 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:27:00.811317 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:27:00.811317 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:27:00.811317 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:27:00.811317 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:27:00.811317 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:27:00.811317 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:27:00.811317 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 17:27:00.811317 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Aug 24 17:27:00.813643 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 57/114] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Aug 24 17:27:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:27:00.819585 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b223d4d8-876a-40af-bb8b-1390d5ae8b6b None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:00.855893 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5735511d-7261-4dc8-b166-be5b15827470 None None] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:00.856241 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5735511d-7261-4dc8-b166-be5b15827470 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:00.856528 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5735511d-7261-4dc8-b166-be5b15827470 None None] PATH_INFO: `/` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:00.858496 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 58/115] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:27:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 17:27:00.866506 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f829212-3355-4ea3-b791-85413f4ab91c None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:00.866798 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f829212-3355-4ea3-b791-85413f4ab91c None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:00.867082 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f829212-3355-4ea3-b791-85413f4ab91c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:00.896179 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-5f829212-3355-4ea3-b791-85413f4ab91c None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:27:00.897459 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-5f829212-3355-4ea3-b791-85413f4ab91c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:27:00.949523 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:00.989877 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b223d4d8-876a-40af-bb8b-1390d5ae8b6b None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:00.999231 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:01.018460 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 59/116] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:27:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 17:27:01.039319 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c85639d-d68c-46b2-a231-295fe41365e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:01.040289 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c85639d-d68c-46b2-a231-295fe41365e5 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:01.040795 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c85639d-d68c-46b2-a231-295fe41365e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:01.041291 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c85639d-d68c-46b2-a231-295fe41365e5 None admin] PATH_INFO: `/v3/roles` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:01.042205 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c85639d-d68c-46b2-a231-295fe41365e5 None admin] RBAC: Adding query filter params (name=member) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:27:01.042862 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c85639d-d68c-46b2-a231-295fe41365e5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:01.044756 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c85639d-d68c-46b2-a231-295fe41365e5 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:01.059250 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 58/117] 10.222.0.45 () {56 vars in 1101 bytes} [Wed Aug 24 17:27:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:27:01.065000 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f829212-3355-4ea3-b791-85413f4ab91c None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:01.106055 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c85639d-d68c-46b2-a231-295fe41365e5 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:01.179482 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f829212-3355-4ea3-b791-85413f4ab91c None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:01.220575 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38019220-4c8d-4ac4-98d0-bc5480c9786e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:01.221488 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38019220-4c8d-4ac4-98d0-bc5480c9786e None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:01.221935 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38019220-4c8d-4ac4-98d0-bc5480c9786e None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:01.222389 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38019220-4c8d-4ac4-98d0-bc5480c9786e None admin] PATH_INFO: `/v3/users/1f7a270ae41e45eab15c82f41826ca94` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:01.224903 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38019220-4c8d-4ac4-98d0-bc5480c9786e None admin] RBAC: Authorizing `identity:get_user(user_id=1f7a270ae41e45eab15c82f41826ca94)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:01.226535 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38019220-4c8d-4ac4-98d0-bc5480c9786e None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:01.232365 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 60/118] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Aug 24 17:27:01 2022] GET /identity/v3/users/1f7a270ae41e45eab15c82f41826ca94 => generated 275 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:27:01.237539 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38019220-4c8d-4ac4-98d0-bc5480c9786e None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:01.272157 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c85639d-d68c-46b2-a231-295fe41365e5 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:01.281424 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:01.311998 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fd6a28-c7f5-446d-93ce-21187842f015 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:01.312771 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fd6a28-c7f5-446d-93ce-21187842f015 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:01.313141 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fd6a28-c7f5-446d-93ce-21187842f015 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:01.313628 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fd6a28-c7f5-446d-93ce-21187842f015 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:01.325377 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fd6a28-c7f5-446d-93ce-21187842f015 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:27:01.331099 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application [None req-a1fd6a28-c7f5-446d-93ce-21187842f015 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:27:01.331099 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:27:01.331099 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:27:01.331099 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:27:01.331099 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:27:01.331099 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:27:01.331099 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:27:01.331099 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:27:01.331099 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:27:01.331099 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:27:01.331099 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:27:01.331099 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:27:01.331099 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:27:01.331099 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:27:01.331099 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:27:01.331099 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:27:01.331964 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:27:01.331964 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:27:01.331964 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:27:01.331964 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:27:01.331964 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:27:01.331964 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:27:01.331964 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:27:01.331964 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:27:01.331964 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:27:01.331964 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:27:01.331964 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:27:01.331964 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application with Lock( Aug 24 17:27:01.331964 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:27:01.331964 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:27:01.331964 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:27:01.331964 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:27:01.332753 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:27:01.332753 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:27:01.332753 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:27:01.332753 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:27:01.332753 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:27:01.332753 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:27:01.332753 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:27:01.332753 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:27:01.332753 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:27:01.332753 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:27:01.332753 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:27:01.332753 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Aug 24 17:27:01.335220 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 59/119] 10.222.0.45 () {56 vars in 1096 bytes} [Wed Aug 24 17:27:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 17:27:01.336869 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89b21a6-ef21-42f4-ac3b-d38a3d08ade0 None None] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:01.337350 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89b21a6-ef21-42f4-ac3b-d38a3d08ade0 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:01.337815 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89b21a6-ef21-42f4-ac3b-d38a3d08ade0 None None] PATH_INFO: `/` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:01.339954 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 60/120] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:27:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:27:01.344932 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89b21a6-ef21-42f4-ac3b-d38a3d08ade0 None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:01.351580 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38019220-4c8d-4ac4-98d0-bc5480c9786e None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:01.376474 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:01.387639 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe99cdd5-7a53-499e-a6b8-70115005e595 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:01.388436 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe99cdd5-7a53-499e-a6b8-70115005e595 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:01.389000 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe99cdd5-7a53-499e-a6b8-70115005e595 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:01.389454 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe99cdd5-7a53-499e-a6b8-70115005e595 None admin] PATH_INFO: `/v3/projects/cc1182fc1781499884cebfaacd1ef5a0` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:01.390807 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe99cdd5-7a53-499e-a6b8-70115005e595 None admin] RBAC: Authorizing `identity:get_project(project_id=cc1182fc1781499884cebfaacd1ef5a0)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:01.392431 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe99cdd5-7a53-499e-a6b8-70115005e595 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:01.394291 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 61/121] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Aug 24 17:27:01 2022] GET /identity/v3/projects/cc1182fc1781499884cebfaacd1ef5a0 => generated 298 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:27:01.395274 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe99cdd5-7a53-499e-a6b8-70115005e595 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:01.460069 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89b21a6-ef21-42f4-ac3b-d38a3d08ade0 None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:01.497943 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5191ba1-bbcf-482c-9bda-0a39bf101707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:01.498680 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5191ba1-bbcf-482c-9bda-0a39bf101707 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:01.499027 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5191ba1-bbcf-482c-9bda-0a39bf101707 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:01.499377 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5191ba1-bbcf-482c-9bda-0a39bf101707 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:01.500114 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe99cdd5-7a53-499e-a6b8-70115005e595 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:01.501982 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:01.513679 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5191ba1-bbcf-482c-9bda-0a39bf101707 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:27:01.518750 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application [None req-e5191ba1-bbcf-482c-9bda-0a39bf101707 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 17:27:01.518750 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:27:01.518750 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:27:01.518750 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:27:01.518750 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:27:01.518750 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:27:01.518750 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:27:01.518750 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:27:01.518750 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:27:01.518750 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:27:01.518750 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:27:01.518750 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:27:01.518750 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:27:01.518750 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:27:01.518750 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:27:01.518750 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:27:01.519603 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:27:01.519603 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:27:01.519603 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:27:01.519603 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:27:01.519603 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:27:01.519603 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:27:01.519603 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:27:01.519603 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:27:01.519603 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:27:01.519603 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:27:01.519603 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:27:01.519603 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application with Lock( Aug 24 17:27:01.519603 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:27:01.519603 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:27:01.519603 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:27:01.519603 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:27:01.520352 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:27:01.520352 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:27:01.520352 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:27:01.520352 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:27:01.520352 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:27:01.520352 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:27:01.520352 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:27:01.520352 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:27:01.520352 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:27:01.520352 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:27:01.520352 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 17:27:01.520352 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Aug 24 17:27:01.520352 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 61/122] 10.222.0.45 () {56 vars in 1114 bytes} [Wed Aug 24 17:27:01 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:27:01.521737 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ba93f1-ddc6-4046-9e72-555777ce2f59 None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:01.522132 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ba93f1-ddc6-4046-9e72-555777ce2f59 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:01.522497 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ba93f1-ddc6-4046-9e72-555777ce2f59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:01.541166 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1916fd-ddfe-4dd8-a2f9-183fd973a1bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:01.541758 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-d2ba93f1-ddc6-4046-9e72-555777ce2f59 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:27:01.542333 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1916fd-ddfe-4dd8-a2f9-183fd973a1bc None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:01.542759 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1916fd-ddfe-4dd8-a2f9-183fd973a1bc None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:01.543200 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1916fd-ddfe-4dd8-a2f9-183fd973a1bc None admin] PATH_INFO: `/v3/roles` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:01.543324 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-d2ba93f1-ddc6-4046-9e72-555777ce2f59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:27:01.544133 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1916fd-ddfe-4dd8-a2f9-183fd973a1bc None admin] RBAC: Adding query filter params (name=admin) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:27:01.544654 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1916fd-ddfe-4dd8-a2f9-183fd973a1bc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:01.546218 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1916fd-ddfe-4dd8-a2f9-183fd973a1bc None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:01.555337 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 62/123] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Aug 24 17:27:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:27:01.556450 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1916fd-ddfe-4dd8-a2f9-183fd973a1bc None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:01.617731 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 62/124] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:27:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:27:01.625488 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ba93f1-ddc6-4046-9e72-555777ce2f59 None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:01.656116 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:01.664576 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1916fd-ddfe-4dd8-a2f9-183fd973a1bc None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:01.709838 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d012971f-dece-47fe-b28e-7a163a280879 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:01.710657 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d012971f-dece-47fe-b28e-7a163a280879 None admin] REQUEST_METHOD: `PUT` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:01.711473 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d012971f-dece-47fe-b28e-7a163a280879 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:01.711762 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d012971f-dece-47fe-b28e-7a163a280879 None admin] PATH_INFO: `/v3/projects/cc1182fc1781499884cebfaacd1ef5a0/users/1f7a270ae41e45eab15c82f41826ca94/roles/02476a5b82eb4b188c18a5d706a3ef68` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:01.717180 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d012971f-dece-47fe-b28e-7a163a280879 None admin] RBAC: Authorizing `identity:create_grant(project_id=cc1182fc1781499884cebfaacd1ef5a0, user_id=1f7a270ae41e45eab15c82f41826ca94, role_id=02476a5b82eb4b188c18a5d706a3ef68)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:01.719805 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d012971f-dece-47fe-b28e-7a163a280879 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:01.748529 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 63/125] 10.222.0.45 () {58 vars in 1439 bytes} [Wed Aug 24 17:27:01 2022] PUT /identity/v3/projects/cc1182fc1781499884cebfaacd1ef5a0/users/1f7a270ae41e45eab15c82f41826ca94/roles/02476a5b82eb4b188c18a5d706a3ef68 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:27:01.750121 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d012971f-dece-47fe-b28e-7a163a280879 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:01.810951 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:01.841397 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ba93f1-ddc6-4046-9e72-555777ce2f59 None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:01.942768 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00331db5-20d2-4b44-9be6-112fba43576f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:01.943792 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00331db5-20d2-4b44-9be6-112fba43576f None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:01.944265 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00331db5-20d2-4b44-9be6-112fba43576f None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:01.944764 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00331db5-20d2-4b44-9be6-112fba43576f None admin] PATH_INFO: `/v3/roles` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:01.945662 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00331db5-20d2-4b44-9be6-112fba43576f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:27:01.946175 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00331db5-20d2-4b44-9be6-112fba43576f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:01.948308 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00331db5-20d2-4b44-9be6-112fba43576f None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:01.961533 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d012971f-dece-47fe-b28e-7a163a280879 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:01.965451 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 63/126] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Aug 24 17:27:01 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 347 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:27:01.966339 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00331db5-20d2-4b44-9be6-112fba43576f None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:02.007805 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0acc8c-b44e-4fce-b3cf-32149997e0b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:02.008465 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0acc8c-b44e-4fce-b3cf-32149997e0b1 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:02.008796 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0acc8c-b44e-4fce-b3cf-32149997e0b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:02.009076 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0acc8c-b44e-4fce-b3cf-32149997e0b1 None admin] PATH_INFO: `/v3/users/cc2669e1aa494dbcb7164d2e5e46378c` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:02.015087 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0acc8c-b44e-4fce-b3cf-32149997e0b1 None admin] RBAC: Authorizing `identity:get_user(user_id=cc2669e1aa494dbcb7164d2e5e46378c)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:02.015087 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0acc8c-b44e-4fce-b3cf-32149997e0b1 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:02.015309 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 64/127] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Aug 24 17:27:01 2022] GET /identity/v3/users/cc2669e1aa494dbcb7164d2e5e46378c => generated 247 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:27:02.016016 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0acc8c-b44e-4fce-b3cf-32149997e0b1 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:02.111613 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:02.141951 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:02.153057 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0acc8c-b44e-4fce-b3cf-32149997e0b1 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:02.159644 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00331db5-20d2-4b44-9be6-112fba43576f None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:02.211235 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8243e566-f07e-473b-8982-f5fbaef82ecb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:02.211806 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8243e566-f07e-473b-8982-f5fbaef82ecb None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:02.212087 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8243e566-f07e-473b-8982-f5fbaef82ecb None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:02.212360 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8243e566-f07e-473b-8982-f5fbaef82ecb None admin] PATH_INFO: `/v3/users/1f7a270ae41e45eab15c82f41826ca94` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:02.213617 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8243e566-f07e-473b-8982-f5fbaef82ecb None admin] RBAC: Authorizing `identity:get_user(user_id=1f7a270ae41e45eab15c82f41826ca94)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:02.215320 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8243e566-f07e-473b-8982-f5fbaef82ecb None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:02.220575 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 65/128] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Aug 24 17:27:02 2022] GET /identity/v3/users/1f7a270ae41e45eab15c82f41826ca94 => generated 275 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:27:02.220575 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8243e566-f07e-473b-8982-f5fbaef82ecb None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:02.241282 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bba301c-2d7c-44f8-b4d5-af277725214a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:02.241947 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bba301c-2d7c-44f8-b4d5-af277725214a None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:02.242233 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bba301c-2d7c-44f8-b4d5-af277725214a None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:02.242515 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bba301c-2d7c-44f8-b4d5-af277725214a None admin] PATH_INFO: `/v3/roles/member` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:02.284964 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bba301c-2d7c-44f8-b4d5-af277725214a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:27:02.294787 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application [None req-4bba301c-2d7c-44f8-b4d5-af277725214a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 17:27:02.294787 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:27:02.294787 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:27:02.294787 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:27:02.294787 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:27:02.294787 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:27:02.294787 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:27:02.294787 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:27:02.294787 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:27:02.294787 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:27:02.294787 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:27:02.294787 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:27:02.294787 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:27:02.294787 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:27:02.294787 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:27:02.294787 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:27:02.295726 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:27:02.295726 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:27:02.295726 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:27:02.295726 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:27:02.295726 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:27:02.295726 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:27:02.295726 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:27:02.295726 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:27:02.295726 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:27:02.295726 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:27:02.295726 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:27:02.295726 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application with Lock( Aug 24 17:27:02.295726 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:27:02.295726 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:27:02.295726 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:27:02.295726 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:27:02.296480 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:27:02.296480 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:27:02.296480 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:27:02.296480 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:27:02.296480 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:27:02.296480 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:27:02.296480 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:27:02.296480 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:27:02.296480 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:27:02.296480 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:27:02.296480 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 17:27:02.296480 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Aug 24 17:27:02.301513 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 64/129] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Aug 24 17:27:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 336 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:27:02.303507 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bba301c-2d7c-44f8-b4d5-af277725214a None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:02.370123 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8243e566-f07e-473b-8982-f5fbaef82ecb None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:02.434220 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feba2010-5766-459f-be7a-6f9704d27e30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:02.434912 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feba2010-5766-459f-be7a-6f9704d27e30 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:02.435203 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feba2010-5766-459f-be7a-6f9704d27e30 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:02.435513 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feba2010-5766-459f-be7a-6f9704d27e30 None admin] PATH_INFO: `/v3/projects/cc1182fc1781499884cebfaacd1ef5a0` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:02.441322 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feba2010-5766-459f-be7a-6f9704d27e30 None admin] RBAC: Authorizing `identity:get_project(project_id=cc1182fc1781499884cebfaacd1ef5a0)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:02.442911 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feba2010-5766-459f-be7a-6f9704d27e30 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:02.444599 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 66/130] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Aug 24 17:27:02 2022] GET /identity/v3/projects/cc1182fc1781499884cebfaacd1ef5a0 => generated 298 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:27:02.445508 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feba2010-5766-459f-be7a-6f9704d27e30 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:02.482853 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bba301c-2d7c-44f8-b4d5-af277725214a None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:02.489193 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:02.565982 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feba2010-5766-459f-be7a-6f9704d27e30 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:02.577256 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05adb5af-ce50-4067-b473-e6f84051fca9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:02.577912 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05adb5af-ce50-4067-b473-e6f84051fca9 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:02.578197 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05adb5af-ce50-4067-b473-e6f84051fca9 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:02.578476 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05adb5af-ce50-4067-b473-e6f84051fca9 None admin] PATH_INFO: `/v3/projects/cc1182fc1781499884cebfaacd1ef5a0` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:02.579665 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05adb5af-ce50-4067-b473-e6f84051fca9 None admin] RBAC: Authorizing `identity:get_project(project_id=cc1182fc1781499884cebfaacd1ef5a0)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:02.581220 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05adb5af-ce50-4067-b473-e6f84051fca9 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:02.582868 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 65/131] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Aug 24 17:27:02 2022] GET /identity/v3/projects/cc1182fc1781499884cebfaacd1ef5a0 => generated 298 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:27:02.583606 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05adb5af-ce50-4067-b473-e6f84051fca9 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:02.606778 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47ad87b-9ba2-40ed-b93d-67a6aedd1d9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:02.607392 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47ad87b-9ba2-40ed-b93d-67a6aedd1d9b None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:02.607668 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47ad87b-9ba2-40ed-b93d-67a6aedd1d9b None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:02.607937 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47ad87b-9ba2-40ed-b93d-67a6aedd1d9b None admin] PATH_INFO: `/v3/roles` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:02.608637 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47ad87b-9ba2-40ed-b93d-67a6aedd1d9b None admin] RBAC: Adding query filter params (name=member) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:27:02.608944 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47ad87b-9ba2-40ed-b93d-67a6aedd1d9b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:02.610323 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47ad87b-9ba2-40ed-b93d-67a6aedd1d9b None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:02.611858 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:02.626126 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 67/132] 10.222.0.45 () {56 vars in 1101 bytes} [Wed Aug 24 17:27:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:27:02.632260 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47ad87b-9ba2-40ed-b93d-67a6aedd1d9b None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:02.743382 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05adb5af-ce50-4067-b473-e6f84051fca9 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:02.751291 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:02.763513 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47ad87b-9ba2-40ed-b93d-67a6aedd1d9b None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:02.787393 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c47a46-6a17-471b-8644-418743850d61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:02.788097 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c47a46-6a17-471b-8644-418743850d61 None admin] REQUEST_METHOD: `PUT` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:02.788805 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c47a46-6a17-471b-8644-418743850d61 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:02.789101 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c47a46-6a17-471b-8644-418743850d61 None admin] PATH_INFO: `/v3/projects/cc1182fc1781499884cebfaacd1ef5a0/users/cc2669e1aa494dbcb7164d2e5e46378c/roles/e411ad9eb96e4e8d85b59b5b700ada7b` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:02.792125 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c47a46-6a17-471b-8644-418743850d61 None admin] RBAC: Authorizing `identity:create_grant(project_id=cc1182fc1781499884cebfaacd1ef5a0, user_id=cc2669e1aa494dbcb7164d2e5e46378c, role_id=e411ad9eb96e4e8d85b59b5b700ada7b)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:02.796896 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c47a46-6a17-471b-8644-418743850d61 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:02.808487 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c46e70c-df42-4d16-bf74-9849e66cc049 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:02.809624 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c46e70c-df42-4d16-bf74-9849e66cc049 None admin] REQUEST_METHOD: `PUT` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:02.810156 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c46e70c-df42-4d16-bf74-9849e66cc049 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:02.810655 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c46e70c-df42-4d16-bf74-9849e66cc049 None admin] PATH_INFO: `/v3/projects/cc1182fc1781499884cebfaacd1ef5a0/users/1f7a270ae41e45eab15c82f41826ca94/roles/148f135e4a134697a032be04bc9faa4c` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:02.813370 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c46e70c-df42-4d16-bf74-9849e66cc049 None admin] RBAC: Authorizing `identity:create_grant(project_id=cc1182fc1781499884cebfaacd1ef5a0, user_id=1f7a270ae41e45eab15c82f41826ca94, role_id=148f135e4a134697a032be04bc9faa4c)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:02.814050 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 66/133] 10.222.0.45 () {58 vars in 1439 bytes} [Wed Aug 24 17:27:02 2022] PUT /identity/v3/projects/cc1182fc1781499884cebfaacd1ef5a0/users/cc2669e1aa494dbcb7164d2e5e46378c/roles/e411ad9eb96e4e8d85b59b5b700ada7b => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:27:02.814773 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c47a46-6a17-471b-8644-418743850d61 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:02.815986 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c46e70c-df42-4d16-bf74-9849e66cc049 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:02.831999 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 68/134] 10.222.0.45 () {58 vars in 1439 bytes} [Wed Aug 24 17:27:02 2022] PUT /identity/v3/projects/cc1182fc1781499884cebfaacd1ef5a0/users/1f7a270ae41e45eab15c82f41826ca94/roles/148f135e4a134697a032be04bc9faa4c => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:27:02.838701 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a04aed-6270-4f83-a78b-a9da446532d2 None None] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:02.839325 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a04aed-6270-4f83-a78b-a9da446532d2 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:02.839809 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a04aed-6270-4f83-a78b-a9da446532d2 None None] PATH_INFO: `/` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:02.841914 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 69/135] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:27:02 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:27:02.848436 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d27377e-6d8e-4038-a337-88a04fd85919 None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:02.849089 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d27377e-6d8e-4038-a337-88a04fd85919 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:02.849626 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d27377e-6d8e-4038-a337-88a04fd85919 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:02.871217 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-6d27377e-6d8e-4038-a337-88a04fd85919 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:27:02.872846 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-6d27377e-6d8e-4038-a337-88a04fd85919 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:27:02.897243 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:02.938145 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c47a46-6a17-471b-8644-418743850d61 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:02.962455 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 70/136] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:27:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:27:02.982192 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdf49ea-1a65-4f3c-b1dc-3751fefef6e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:02.982853 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdf49ea-1a65-4f3c-b1dc-3751fefef6e3 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:02.983139 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdf49ea-1a65-4f3c-b1dc-3751fefef6e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:02.983418 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdf49ea-1a65-4f3c-b1dc-3751fefef6e3 None admin] PATH_INFO: `/v3/users/1f7a270ae41e45eab15c82f41826ca94` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:02.984639 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdf49ea-1a65-4f3c-b1dc-3751fefef6e3 None admin] RBAC: Authorizing `identity:get_user(user_id=1f7a270ae41e45eab15c82f41826ca94)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:02.986177 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdf49ea-1a65-4f3c-b1dc-3751fefef6e3 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:02.988096 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 67/137] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Aug 24 17:27:02 2022] GET /identity/v3/users/1f7a270ae41e45eab15c82f41826ca94 => generated 275 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:27:02.993952 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d27377e-6d8e-4038-a337-88a04fd85919 None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:03.112996 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:03.141027 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d27377e-6d8e-4038-a337-88a04fd85919 None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:03.180459 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d18bee-7769-45a4-831e-ffb1367df03c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:03.181161 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d18bee-7769-45a4-831e-ffb1367df03c None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:03.181399 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d18bee-7769-45a4-831e-ffb1367df03c None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:03.181673 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d18bee-7769-45a4-831e-ffb1367df03c None admin] PATH_INFO: `/v3/projects/e615e00f222d4a298a1f7ceea74eb618` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:03.183407 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d18bee-7769-45a4-831e-ffb1367df03c None admin] RBAC: Authorizing `identity:get_project(project_id=e615e00f222d4a298a1f7ceea74eb618)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:03.184877 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d18bee-7769-45a4-831e-ffb1367df03c None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:03.186685 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 71/138] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Aug 24 17:27:02 2022] GET /identity/v3/projects/e615e00f222d4a298a1f7ceea74eb618 => generated 312 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:27:03.192371 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdf49ea-1a65-4f3c-b1dc-3751fefef6e3 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:03.294653 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d18bee-7769-45a4-831e-ffb1367df03c None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:03.308685 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdf49ea-1a65-4f3c-b1dc-3751fefef6e3 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:03.376599 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9e180e-1a1a-4c69-bd34-0a3d225be495 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:03.377286 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9e180e-1a1a-4c69-bd34-0a3d225be495 None admin] REQUEST_METHOD: `PUT` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:03.377563 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9e180e-1a1a-4c69-bd34-0a3d225be495 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:03.377837 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9e180e-1a1a-4c69-bd34-0a3d225be495 None admin] PATH_INFO: `/v3/projects/e615e00f222d4a298a1f7ceea74eb618/users/1f7a270ae41e45eab15c82f41826ca94/roles/02476a5b82eb4b188c18a5d706a3ef68` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:03.379885 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9e180e-1a1a-4c69-bd34-0a3d225be495 None admin] RBAC: Authorizing `identity:create_grant(project_id=e615e00f222d4a298a1f7ceea74eb618, user_id=1f7a270ae41e45eab15c82f41826ca94, role_id=02476a5b82eb4b188c18a5d706a3ef68)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:03.381380 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9e180e-1a1a-4c69-bd34-0a3d225be495 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:03.384678 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:03.410757 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 68/139] 10.222.0.45 () {58 vars in 1439 bytes} [Wed Aug 24 17:27:03 2022] PUT /identity/v3/projects/e615e00f222d4a298a1f7ceea74eb618/users/1f7a270ae41e45eab15c82f41826ca94/roles/02476a5b82eb4b188c18a5d706a3ef68 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:27:03.412087 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b06324-eb0e-46f1-a65c-1c5aa0c162a9 None None] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:03.412383 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b06324-eb0e-46f1-a65c-1c5aa0c162a9 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:03.412689 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b06324-eb0e-46f1-a65c-1c5aa0c162a9 None None] PATH_INFO: `/` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:03.422316 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 69/140] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:27:03 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:27:03.423757 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f2f4fb-c5ff-4338-982d-7c0fe6ee628c None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:03.424053 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f2f4fb-c5ff-4338-982d-7c0fe6ee628c None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:03.424334 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f2f4fb-c5ff-4338-982d-7c0fe6ee628c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:03.456356 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-76f2f4fb-c5ff-4338-982d-7c0fe6ee628c None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:27:03.459678 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-76f2f4fb-c5ff-4338-982d-7c0fe6ee628c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:27:03.480939 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d18bee-7769-45a4-831e-ffb1367df03c None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:03.534893 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 70/141] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:27:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:27:03.547351 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b9a279-ced3-4c14-87a9-240b4a5f29db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:03.548362 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b9a279-ced3-4c14-87a9-240b4a5f29db None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:03.548863 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b9a279-ced3-4c14-87a9-240b4a5f29db None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:03.549415 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b9a279-ced3-4c14-87a9-240b4a5f29db None admin] PATH_INFO: `/v3/domains/default` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:03.553252 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b9a279-ced3-4c14-87a9-240b4a5f29db None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:03.554994 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b9a279-ced3-4c14-87a9-240b4a5f29db None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:03.557471 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 72/142] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 17:27:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:27:03.563299 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f2f4fb-c5ff-4338-982d-7c0fe6ee628c None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:03.685927 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f2f4fb-c5ff-4338-982d-7c0fe6ee628c None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:03.713778 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:03.723070 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549b636d-d375-4587-9482-5eb89d842e7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:03.723703 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549b636d-d375-4587-9482-5eb89d842e7d None admin] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:03.723989 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549b636d-d375-4587-9482-5eb89d842e7d None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:03.724331 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549b636d-d375-4587-9482-5eb89d842e7d None admin] PATH_INFO: `/v3/users` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:03.725037 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549b636d-d375-4587-9482-5eb89d842e7d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:03.726709 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549b636d-d375-4587-9482-5eb89d842e7d None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:03.755930 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 71/143] 10.222.0.45 () {60 vars in 1132 bytes} [Wed Aug 24 17:27:03 2022] POST /identity/v3/users => generated 283 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:27:03.998049 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b9a279-ced3-4c14-87a9-240b4a5f29db None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:04.024016 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:04.126386 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b9a279-ced3-4c14-87a9-240b4a5f29db None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:04.164876 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5218d9c4-826d-4de7-8da4-b924cacfd7d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:04.165756 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5218d9c4-826d-4de7-8da4-b924cacfd7d0 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:04.166183 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5218d9c4-826d-4de7-8da4-b924cacfd7d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:04.167090 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5218d9c4-826d-4de7-8da4-b924cacfd7d0 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:04.178160 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5218d9c4-826d-4de7-8da4-b924cacfd7d0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:27:04.183691 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application [None req-5218d9c4-826d-4de7-8da4-b924cacfd7d0 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 17:27:04.183691 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:27:04.183691 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:27:04.183691 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:27:04.183691 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:27:04.183691 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:27:04.183691 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:27:04.183691 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:27:04.183691 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:27:04.183691 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:27:04.183691 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:27:04.183691 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:27:04.183691 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:27:04.183691 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:27:04.183691 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:27:04.183691 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:27:04.185145 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:27:04.185145 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:27:04.185145 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:27:04.185145 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:27:04.185145 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:27:04.185145 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:27:04.185145 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:27:04.185145 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:27:04.185145 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:27:04.185145 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:27:04.185145 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:27:04.185145 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application with Lock( Aug 24 17:27:04.185145 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:27:04.185145 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:27:04.185145 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:27:04.185145 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:27:04.186097 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:27:04.186097 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:27:04.186097 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:27:04.186097 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:27:04.186097 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:27:04.186097 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:27:04.186097 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:27:04.186097 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:27:04.186097 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:27:04.186097 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:27:04.186097 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 17:27:04.186097 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Aug 24 17:27:04.187915 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 73/144] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Aug 24 17:27:03 2022] GET /identity/v3/roles/reader => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:27:04.193706 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549b636d-d375-4587-9482-5eb89d842e7d None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:04.303183 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549b636d-d375-4587-9482-5eb89d842e7d None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:04.349338 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6bf384-d553-443e-9c73-0e7c8b2a88ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:04.350210 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6bf384-d553-443e-9c73-0e7c8b2a88ad None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:04.350645 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6bf384-d553-443e-9c73-0e7c8b2a88ad None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:04.351064 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6bf384-d553-443e-9c73-0e7c8b2a88ad None admin] PATH_INFO: `/v3/roles` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:04.352049 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6bf384-d553-443e-9c73-0e7c8b2a88ad None admin] RBAC: Adding query filter params (name=reader) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:27:04.352836 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6bf384-d553-443e-9c73-0e7c8b2a88ad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:04.354460 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6bf384-d553-443e-9c73-0e7c8b2a88ad None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:04.363803 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 72/145] 10.222.0.45 () {56 vars in 1101 bytes} [Wed Aug 24 17:27:04 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:27:04.369503 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5218d9c4-826d-4de7-8da4-b924cacfd7d0 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:04.464910 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:04.495132 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5218d9c4-826d-4de7-8da4-b924cacfd7d0 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:04.534221 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce10a3c-b89b-492a-a5a4-714c5e5eed08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:04.534918 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce10a3c-b89b-492a-a5a4-714c5e5eed08 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:04.535206 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce10a3c-b89b-492a-a5a4-714c5e5eed08 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:04.535485 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce10a3c-b89b-492a-a5a4-714c5e5eed08 None admin] PATH_INFO: `/v3/users/9bac15d3c1cc40f9bbfa687de74340b4` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:04.536669 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce10a3c-b89b-492a-a5a4-714c5e5eed08 None admin] RBAC: Authorizing `identity:get_user(user_id=9bac15d3c1cc40f9bbfa687de74340b4)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:04.538125 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce10a3c-b89b-492a-a5a4-714c5e5eed08 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:04.539877 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 74/146] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Aug 24 17:27:04 2022] GET /identity/v3/users/9bac15d3c1cc40f9bbfa687de74340b4 => generated 289 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:27:04.546937 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6bf384-d553-443e-9c73-0e7c8b2a88ad None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:04.606938 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:04.674310 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6bf384-d553-443e-9c73-0e7c8b2a88ad None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:04.715281 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c65ce85-760a-4b6a-bf25-b55e4e09b61a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:04.715843 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c65ce85-760a-4b6a-bf25-b55e4e09b61a None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:04.716123 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c65ce85-760a-4b6a-bf25-b55e4e09b61a None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:04.716393 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c65ce85-760a-4b6a-bf25-b55e4e09b61a None admin] PATH_INFO: `/v3/projects/cc1182fc1781499884cebfaacd1ef5a0` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:04.721383 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c65ce85-760a-4b6a-bf25-b55e4e09b61a None admin] RBAC: Authorizing `identity:get_project(project_id=cc1182fc1781499884cebfaacd1ef5a0)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:04.722859 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c65ce85-760a-4b6a-bf25-b55e4e09b61a None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:04.724612 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 73/147] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Aug 24 17:27:04 2022] GET /identity/v3/projects/cc1182fc1781499884cebfaacd1ef5a0 => generated 298 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:27:04.730801 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce10a3c-b89b-492a-a5a4-714c5e5eed08 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:04.856250 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce10a3c-b89b-492a-a5a4-714c5e5eed08 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:04.896443 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170acdf3-be0e-4f53-bc80-2b74fa01dec1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:04.897154 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170acdf3-be0e-4f53-bc80-2b74fa01dec1 None admin] REQUEST_METHOD: `PUT` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:04.897438 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170acdf3-be0e-4f53-bc80-2b74fa01dec1 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:04.897716 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170acdf3-be0e-4f53-bc80-2b74fa01dec1 None admin] PATH_INFO: `/v3/projects/cc1182fc1781499884cebfaacd1ef5a0/users/9bac15d3c1cc40f9bbfa687de74340b4/roles/645db6d227604357b1949889630755ff` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:04.899669 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170acdf3-be0e-4f53-bc80-2b74fa01dec1 None admin] RBAC: Authorizing `identity:create_grant(project_id=cc1182fc1781499884cebfaacd1ef5a0, user_id=9bac15d3c1cc40f9bbfa687de74340b4, role_id=645db6d227604357b1949889630755ff)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:04.901249 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170acdf3-be0e-4f53-bc80-2b74fa01dec1 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:04.917146 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba6bab4-d03b-4ea2-a41b-b70d7cfe09a0 None None] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:04.917440 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba6bab4-d03b-4ea2-a41b-b70d7cfe09a0 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:04.917703 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba6bab4-d03b-4ea2-a41b-b70d7cfe09a0 None None] PATH_INFO: `/` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:04.919198 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 74/148] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:27:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 17:27:04.920059 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 75/149] 10.222.0.45 () {58 vars in 1439 bytes} [Wed Aug 24 17:27:04 2022] PUT /identity/v3/projects/cc1182fc1781499884cebfaacd1ef5a0/users/9bac15d3c1cc40f9bbfa687de74340b4/roles/645db6d227604357b1949889630755ff => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:27:04.925276 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287b4df1-29b1-4261-99af-f60a8b544572 None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:04.925572 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287b4df1-29b1-4261-99af-f60a8b544572 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:04.925900 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287b4df1-29b1-4261-99af-f60a8b544572 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:04.946823 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-287b4df1-29b1-4261-99af-f60a8b544572 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:27:04.948650 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-287b4df1-29b1-4261-99af-f60a8b544572 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:27:05.034412 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 75/150] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:27:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:27:05.608402 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170acdf3-be0e-4f53-bc80-2b74fa01dec1 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:05.612595 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:05.695314 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53f5675-174d-45e0-b7d4-12a062a1fdb2 None None] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:05.695809 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53f5675-174d-45e0-b7d4-12a062a1fdb2 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:05.695879 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53f5675-174d-45e0-b7d4-12a062a1fdb2 None None] PATH_INFO: `/` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:05.697227 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 76/151] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:27:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 17:27:05.704663 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860cfbf6-bef0-4405-b480-309a01f388cc None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:05.705008 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860cfbf6-bef0-4405-b480-309a01f388cc None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:05.705297 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860cfbf6-bef0-4405-b480-309a01f388cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:05.739368 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-860cfbf6-bef0-4405-b480-309a01f388cc None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:27:05.743288 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-860cfbf6-bef0-4405-b480-309a01f388cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:27:05.779328 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170acdf3-be0e-4f53-bc80-2b74fa01dec1 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:05.821591 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:05.855877 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb5644a-7fcc-4d7b-b5a8-562aeffb46b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:05.856524 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb5644a-7fcc-4d7b-b5a8-562aeffb46b2 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:05.856839 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb5644a-7fcc-4d7b-b5a8-562aeffb46b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:05.857118 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb5644a-7fcc-4d7b-b5a8-562aeffb46b2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:05.874591 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 77/152] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:27:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:27:05.875643 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134dc66e-c09f-4afb-8e5a-83d3297385d3 None None] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:05.876000 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134dc66e-c09f-4afb-8e5a-83d3297385d3 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:05.878460 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134dc66e-c09f-4afb-8e5a-83d3297385d3 None None] PATH_INFO: `/` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:05.879907 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 78/153] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:27:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:27:05.880707 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb5644a-7fcc-4d7b-b5a8-562aeffb46b2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:27:05.886277 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application [None req-1cb5644a-7fcc-4d7b-b5a8-562aeffb46b2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 17:27:05.886277 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:27:05.886277 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:27:05.886277 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:27:05.886277 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:27:05.886277 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:27:05.886277 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:27:05.886277 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:27:05.886277 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:27:05.886277 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:27:05.886277 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:27:05.886277 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:27:05.886277 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:27:05.886277 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:27:05.886277 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:27:05.886277 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:27:05.887154 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:27:05.887154 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:27:05.887154 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:27:05.887154 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:27:05.887154 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:27:05.887154 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:27:05.887154 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:27:05.887154 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:27:05.887154 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:27:05.887154 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:27:05.887154 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:27:05.887154 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application with Lock( Aug 24 17:27:05.887154 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:27:05.887154 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:27:05.887154 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:27:05.887154 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:27:05.887896 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:27:05.887896 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:27:05.887896 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:27:05.887896 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:27:05.887896 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:27:05.887896 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:27:05.887896 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:27:05.887896 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:27:05.887896 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:27:05.887896 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:27:05.887896 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 17:27:05.887896 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Aug 24 17:27:05.887896 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 76/154] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Aug 24 17:27:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:27:05.889208 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaa93b2-e03c-413b-9a6b-f8b396bfa373 None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:05.889208 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaa93b2-e03c-413b-9a6b-f8b396bfa373 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:05.889208 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaa93b2-e03c-413b-9a6b-f8b396bfa373 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:05.900113 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb5644a-7fcc-4d7b-b5a8-562aeffb46b2 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:05.924571 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-2aaa93b2-e03c-413b-9a6b-f8b396bfa373 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:27:05.927086 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-2aaa93b2-e03c-413b-9a6b-f8b396bfa373 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:27:06.028967 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:06.033462 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb5644a-7fcc-4d7b-b5a8-562aeffb46b2 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:06.065899 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 79/155] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:27:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:27:06.078786 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d473f9f0-0c9c-47e8-95b5-5b4de52b7840 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:06.079404 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d473f9f0-0c9c-47e8-95b5-5b4de52b7840 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:06.079743 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d473f9f0-0c9c-47e8-95b5-5b4de52b7840 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:06.080069 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d473f9f0-0c9c-47e8-95b5-5b4de52b7840 None admin] PATH_INFO: `/v3/roles` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:06.080777 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d473f9f0-0c9c-47e8-95b5-5b4de52b7840 None admin] RBAC: Adding query filter params (name=member) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:27:06.081081 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d473f9f0-0c9c-47e8-95b5-5b4de52b7840 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:06.082527 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d473f9f0-0c9c-47e8-95b5-5b4de52b7840 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:06.091512 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 77/156] 10.222.0.45 () {56 vars in 1101 bytes} [Wed Aug 24 17:27:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:27:06.097236 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aaa93b2-e03c-413b-9a6b-f8b396bfa373 None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:06.191806 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa82d4f-5005-4765-9851-d41de3e7bfc6 None None] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:06.192131 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa82d4f-5005-4765-9851-d41de3e7bfc6 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:06.192373 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa82d4f-5005-4765-9851-d41de3e7bfc6 None None] PATH_INFO: `/` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:06.193752 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 78/157] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:27:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 17:27:06.204669 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ef3e46-1e61-426a-bc71-88f5d708f328 None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:06.205005 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ef3e46-1e61-426a-bc71-88f5d708f328 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:06.205286 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ef3e46-1e61-426a-bc71-88f5d708f328 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:06.231974 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-72ef3e46-1e61-426a-bc71-88f5d708f328 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:27:06.233204 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-72ef3e46-1e61-426a-bc71-88f5d708f328 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:27:06.288027 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aaa93b2-e03c-413b-9a6b-f8b396bfa373 None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:06.291770 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:06.345070 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 79/158] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:27:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Aug 24 17:27:06.361281 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05f6989-a50e-467e-b736-dcf636220d4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:06.361941 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05f6989-a50e-467e-b736-dcf636220d4b None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:06.362218 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05f6989-a50e-467e-b736-dcf636220d4b None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:06.362490 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05f6989-a50e-467e-b736-dcf636220d4b None admin] PATH_INFO: `/v3/users/1f7a270ae41e45eab15c82f41826ca94` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:06.363671 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05f6989-a50e-467e-b736-dcf636220d4b None admin] RBAC: Authorizing `identity:get_user(user_id=1f7a270ae41e45eab15c82f41826ca94)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:06.365152 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05f6989-a50e-467e-b736-dcf636220d4b None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:06.366908 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 80/159] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Aug 24 17:27:06 2022] GET /identity/v3/users/1f7a270ae41e45eab15c82f41826ca94 => generated 275 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:27:06.372659 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ef3e46-1e61-426a-bc71-88f5d708f328 None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:06.449098 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:06.491457 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ef3e46-1e61-426a-bc71-88f5d708f328 None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:06.529534 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f1218b-be2c-4a6a-85e8-a6d36a103cdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:06.530319 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f1218b-be2c-4a6a-85e8-a6d36a103cdc None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:06.530730 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f1218b-be2c-4a6a-85e8-a6d36a103cdc None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:06.531102 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f1218b-be2c-4a6a-85e8-a6d36a103cdc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:06.576312 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05f6989-a50e-467e-b736-dcf636220d4b None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:06.601608 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f1218b-be2c-4a6a-85e8-a6d36a103cdc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:06.603394 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f1218b-be2c-4a6a-85e8-a6d36a103cdc None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:06.668950 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:06.712997 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 80/160] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Aug 24 17:27:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 17:27:06.713893 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f1218b-be2c-4a6a-85e8-a6d36a103cdc None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:06.778031 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05f6989-a50e-467e-b736-dcf636220d4b None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:06.844350 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5199cfad-b68b-4a06-bae9-767fae3b0c08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:06.849068 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5199cfad-b68b-4a06-bae9-767fae3b0c08 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:06.849351 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5199cfad-b68b-4a06-bae9-767fae3b0c08 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:06.849626 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5199cfad-b68b-4a06-bae9-767fae3b0c08 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:06.868678 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5199cfad-b68b-4a06-bae9-767fae3b0c08 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:27:06.873924 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application [None req-5199cfad-b68b-4a06-bae9-767fae3b0c08 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 17:27:06.873924 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:27:06.873924 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:27:06.873924 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:27:06.873924 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:27:06.873924 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:27:06.873924 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:27:06.873924 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:27:06.873924 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:27:06.873924 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:27:06.873924 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:27:06.873924 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:27:06.873924 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:27:06.873924 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:27:06.873924 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:27:06.873924 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:27:06.874827 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:27:06.874827 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:27:06.874827 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:27:06.874827 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:27:06.874827 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:27:06.874827 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:27:06.874827 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:27:06.874827 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:27:06.874827 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:27:06.874827 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:27:06.874827 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:27:06.874827 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application with Lock( Aug 24 17:27:06.874827 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:27:06.874827 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:27:06.874827 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:27:06.874827 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:27:06.875582 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:27:06.875582 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:27:06.875582 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:27:06.875582 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:27:06.875582 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:27:06.875582 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:27:06.875582 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:27:06.875582 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:27:06.875582 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:27:06.875582 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:27:06.875582 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 17:27:06.875582 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Aug 24 17:27:06.880569 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f1218b-be2c-4a6a-85e8-a6d36a103cdc None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:06.880690 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 81/161] 10.222.0.45 () {56 vars in 1114 bytes} [Wed Aug 24 17:27:06 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:27:06.880690 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5199cfad-b68b-4a06-bae9-767fae3b0c08 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:06.917172 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b75d0fe-8c7c-453c-8c86-3d08fa1cb15b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:06.917813 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b75d0fe-8c7c-453c-8c86-3d08fa1cb15b None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:06.918349 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b75d0fe-8c7c-453c-8c86-3d08fa1cb15b None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:06.918622 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b75d0fe-8c7c-453c-8c86-3d08fa1cb15b None admin] PATH_INFO: `/v3/roles/member` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:06.930934 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b75d0fe-8c7c-453c-8c86-3d08fa1cb15b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:27:06.936369 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application [None req-4b75d0fe-8c7c-453c-8c86-3d08fa1cb15b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 17:27:06.936369 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:27:06.936369 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:27:06.936369 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:27:06.936369 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:27:06.936369 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:27:06.936369 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:27:06.936369 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:27:06.936369 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:27:06.936369 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:27:06.936369 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:27:06.936369 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:27:06.936369 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:27:06.936369 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:27:06.936369 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:27:06.936369 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:27:06.937312 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:27:06.937312 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:27:06.937312 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:27:06.937312 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:27:06.937312 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:27:06.937312 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:27:06.937312 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:27:06.937312 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:27:06.937312 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:27:06.937312 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:27:06.937312 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:27:06.937312 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application with Lock( Aug 24 17:27:06.937312 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:27:06.937312 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:27:06.937312 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:27:06.937312 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:27:06.938941 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:27:06.938941 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:27:06.938941 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:27:06.938941 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:27:06.938941 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:27:06.938941 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:27:06.938941 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:27:06.938941 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:27:06.938941 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:27:06.938941 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:27:06.938941 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 17:27:06.938941 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Aug 24 17:27:06.938941 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 81/162] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Aug 24 17:27:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:27:06.940690 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b75d0fe-8c7c-453c-8c86-3d08fa1cb15b None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:06.944572 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:06.965018 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:07.075310 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5199cfad-b68b-4a06-bae9-767fae3b0c08 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:07.102780 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b75d0fe-8c7c-453c-8c86-3d08fa1cb15b None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:07.128340 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d2637f-ed70-4a84-aae5-adb80941c003 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:07.129374 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d2637f-ed70-4a84-aae5-adb80941c003 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:07.129867 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d2637f-ed70-4a84-aae5-adb80941c003 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:07.130349 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d2637f-ed70-4a84-aae5-adb80941c003 None admin] PATH_INFO: `/v3/projects/cc1182fc1781499884cebfaacd1ef5a0` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:07.136573 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d2637f-ed70-4a84-aae5-adb80941c003 None admin] RBAC: Authorizing `identity:get_project(project_id=cc1182fc1781499884cebfaacd1ef5a0)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:07.136573 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d2637f-ed70-4a84-aae5-adb80941c003 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:07.136573 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 82/163] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Aug 24 17:27:06 2022] GET /identity/v3/projects/cc1182fc1781499884cebfaacd1ef5a0 => generated 298 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:27:07.137594 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d2637f-ed70-4a84-aae5-adb80941c003 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:07.207059 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4b7dc9-8e0c-40f8-b03a-1c0e8ba9df46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:07.207686 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4b7dc9-8e0c-40f8-b03a-1c0e8ba9df46 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:07.207964 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4b7dc9-8e0c-40f8-b03a-1c0e8ba9df46 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:07.208236 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4b7dc9-8e0c-40f8-b03a-1c0e8ba9df46 None admin] PATH_INFO: `/v3/roles` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:07.208947 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4b7dc9-8e0c-40f8-b03a-1c0e8ba9df46 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:27:07.209246 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4b7dc9-8e0c-40f8-b03a-1c0e8ba9df46 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:07.210728 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4b7dc9-8e0c-40f8-b03a-1c0e8ba9df46 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:07.227188 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 82/164] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Aug 24 17:27:06 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:27:07.228235 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4b7dc9-8e0c-40f8-b03a-1c0e8ba9df46 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:07.320936 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:07.325247 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d2637f-ed70-4a84-aae5-adb80941c003 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:07.387459 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f00215-d761-4938-b620-5a89b61060f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:07.388697 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f00215-d761-4938-b620-5a89b61060f6 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:07.389191 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f00215-d761-4938-b620-5a89b61060f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:07.389655 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f00215-d761-4938-b620-5a89b61060f6 None admin] PATH_INFO: `/v3/roles` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:07.390541 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f00215-d761-4938-b620-5a89b61060f6 None admin] RBAC: Adding query filter params (name=member) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:27:07.391094 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f00215-d761-4938-b620-5a89b61060f6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:07.400861 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f00215-d761-4938-b620-5a89b61060f6 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:07.410201 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 83/165] 10.222.0.45 () {56 vars in 1101 bytes} [Wed Aug 24 17:27:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:27:07.411337 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f00215-d761-4938-b620-5a89b61060f6 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:07.417171 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:07.461689 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4b7dc9-8e0c-40f8-b03a-1c0e8ba9df46 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:07.520969 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0d1a23-406a-436b-a116-2b65a88c777f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:07.520969 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0d1a23-406a-436b-a116-2b65a88c777f None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:07.520969 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0d1a23-406a-436b-a116-2b65a88c777f None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:07.520969 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0d1a23-406a-436b-a116-2b65a88c777f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:07.533571 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f00215-d761-4938-b620-5a89b61060f6 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:07.551508 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0d1a23-406a-436b-a116-2b65a88c777f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:27:07.561063 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application [None req-6c0d1a23-406a-436b-a116-2b65a88c777f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:27:07.561063 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:27:07.561063 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:27:07.561063 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:27:07.561063 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:27:07.561063 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:27:07.561063 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:27:07.561063 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:27:07.561063 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:27:07.561063 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:27:07.561063 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:27:07.561063 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:27:07.561063 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:27:07.561063 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:27:07.561063 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:27:07.561063 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:27:07.561977 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:27:07.561977 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:27:07.561977 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:27:07.561977 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:27:07.561977 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:27:07.561977 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:27:07.561977 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:27:07.561977 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:27:07.561977 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:27:07.561977 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:27:07.561977 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:27:07.561977 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application with Lock( Aug 24 17:27:07.561977 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:27:07.561977 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:27:07.561977 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:27:07.561977 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:27:07.562735 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:27:07.562735 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:27:07.562735 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:27:07.562735 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:27:07.562735 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:27:07.562735 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:27:07.562735 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:27:07.562735 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:27:07.562735 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:27:07.562735 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:27:07.562735 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:27:07.562735 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Aug 24 17:27:07.562735 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 83/166] 10.222.0.45 () {56 vars in 1096 bytes} [Wed Aug 24 17:27:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 335 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:27:07.563384 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0d1a23-406a-436b-a116-2b65a88c777f None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:07.606684 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fec48b-e130-4471-83ce-242c2f9b0f0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:07.607373 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fec48b-e130-4471-83ce-242c2f9b0f0b None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:07.607661 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fec48b-e130-4471-83ce-242c2f9b0f0b None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:07.607943 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fec48b-e130-4471-83ce-242c2f9b0f0b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:07.608708 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fec48b-e130-4471-83ce-242c2f9b0f0b None admin] RBAC: Adding query filter params (role.id=02476a5b82eb4b188c18a5d706a3ef68, scope.project.id=cc1182fc1781499884cebfaacd1ef5a0, user.id=1f7a270ae41e45eab15c82f41826ca94) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:27:07.609164 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fec48b-e130-4471-83ce-242c2f9b0f0b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:07.610880 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fec48b-e130-4471-83ce-242c2f9b0f0b None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:07.627743 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 84/167] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Aug 24 17:27:07 2022] GET /identity/v3/role_assignments?user.id=1f7a270ae41e45eab15c82f41826ca94&scope.project.id=cc1182fc1781499884cebfaacd1ef5a0&role.id=02476a5b82eb4b188c18a5d706a3ef68 => generated 608 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:27:07.628936 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fec48b-e130-4471-83ce-242c2f9b0f0b None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:07.677848 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:07.793110 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0d1a23-406a-436b-a116-2b65a88c777f None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:07.839074 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fec48b-e130-4471-83ce-242c2f9b0f0b None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:07.881354 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebe41b0-b8a9-49e1-83da-53a6b490e5bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:07.882019 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebe41b0-b8a9-49e1-83da-53a6b490e5bc None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:07.882300 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebe41b0-b8a9-49e1-83da-53a6b490e5bc None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:07.882572 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebe41b0-b8a9-49e1-83da-53a6b490e5bc None admin] PATH_INFO: `/v3/users/1f7a270ae41e45eab15c82f41826ca94` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:07.888698 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebe41b0-b8a9-49e1-83da-53a6b490e5bc None admin] RBAC: Authorizing `identity:get_user(user_id=1f7a270ae41e45eab15c82f41826ca94)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:07.888698 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebe41b0-b8a9-49e1-83da-53a6b490e5bc None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:07.888698 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 84/168] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Aug 24 17:27:07 2022] GET /identity/v3/users/1f7a270ae41e45eab15c82f41826ca94 => generated 275 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:27:07.888698 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebe41b0-b8a9-49e1-83da-53a6b490e5bc None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:07.892383 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e7add2-d693-4607-b769-f3806a4bbc19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:07.893372 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e7add2-d693-4607-b769-f3806a4bbc19 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:07.893950 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e7add2-d693-4607-b769-f3806a4bbc19 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:07.894421 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e7add2-d693-4607-b769-f3806a4bbc19 None admin] PATH_INFO: `/v3/users/1f7a270ae41e45eab15c82f41826ca94` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:07.895774 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e7add2-d693-4607-b769-f3806a4bbc19 None admin] RBAC: Authorizing `identity:get_user(user_id=1f7a270ae41e45eab15c82f41826ca94)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:07.897521 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e7add2-d693-4607-b769-f3806a4bbc19 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:07.899827 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 85/169] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Aug 24 17:27:07 2022] GET /identity/v3/users/1f7a270ae41e45eab15c82f41826ca94 => generated 275 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:27:07.901485 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e7add2-d693-4607-b769-f3806a4bbc19 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:07.908940 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:08.087143 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebe41b0-b8a9-49e1-83da-53a6b490e5bc None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:08.106126 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:08.121975 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e7add2-d693-4607-b769-f3806a4bbc19 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:08.148316 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d48bcd-0edd-4b8d-bd61-d8c8301321a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:08.150502 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d48bcd-0edd-4b8d-bd61-d8c8301321a3 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:08.151051 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d48bcd-0edd-4b8d-bd61-d8c8301321a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:08.152306 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d48bcd-0edd-4b8d-bd61-d8c8301321a3 None admin] PATH_INFO: `/v3/roles` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:08.153227 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d48bcd-0edd-4b8d-bd61-d8c8301321a3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:27:08.156411 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d48bcd-0edd-4b8d-bd61-d8c8301321a3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:08.158109 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d48bcd-0edd-4b8d-bd61-d8c8301321a3 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:08.170554 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 85/170] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Aug 24 17:27:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:27:08.171400 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d48bcd-0edd-4b8d-bd61-d8c8301321a3 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:08.195765 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fd549f-5ea4-443d-9cba-0a4833f5e292 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:08.196383 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fd549f-5ea4-443d-9cba-0a4833f5e292 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:08.196685 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fd549f-5ea4-443d-9cba-0a4833f5e292 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:08.196970 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fd549f-5ea4-443d-9cba-0a4833f5e292 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:08.309285 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:08.349469 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fd549f-5ea4-443d-9cba-0a4833f5e292 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:08.350955 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fd549f-5ea4-443d-9cba-0a4833f5e292 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:08.353495 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d48bcd-0edd-4b8d-bd61-d8c8301321a3 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:08.411218 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cac1a61-0ba8-4a64-8721-95363c97fd1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:08.411903 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cac1a61-0ba8-4a64-8721-95363c97fd1a None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:08.412188 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cac1a61-0ba8-4a64-8721-95363c97fd1a None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:08.412461 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cac1a61-0ba8-4a64-8721-95363c97fd1a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:08.475596 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:08.487298 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 86/171] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Aug 24 17:27:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 587 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 17:27:08.488566 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1390b7cf-7913-4050-8b7c-73e0d8cc81c2 None None] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:08.488856 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1390b7cf-7913-4050-8b7c-73e0d8cc81c2 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:08.489131 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1390b7cf-7913-4050-8b7c-73e0d8cc81c2 None None] PATH_INFO: `/` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:08.490382 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 87/172] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:27:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:27:08.491279 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1390b7cf-7913-4050-8b7c-73e0d8cc81c2 None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:08.495842 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cac1a61-0ba8-4a64-8721-95363c97fd1a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:08.497400 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cac1a61-0ba8-4a64-8721-95363c97fd1a None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:08.607293 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1390b7cf-7913-4050-8b7c-73e0d8cc81c2 None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:08.624907 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:08.647686 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab99ef9-78b3-49a2-bf57-7e85ab65eca4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:08.648673 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab99ef9-78b3-49a2-bf57-7e85ab65eca4 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:08.648959 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab99ef9-78b3-49a2-bf57-7e85ab65eca4 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:08.649235 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab99ef9-78b3-49a2-bf57-7e85ab65eca4 None admin] PATH_INFO: `/v3/users/cc2669e1aa494dbcb7164d2e5e46378c` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:08.650586 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab99ef9-78b3-49a2-bf57-7e85ab65eca4 None admin] RBAC: Authorizing `identity:get_user(user_id=cc2669e1aa494dbcb7164d2e5e46378c)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:08.652513 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab99ef9-78b3-49a2-bf57-7e85ab65eca4 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:08.654264 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 88/173] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Aug 24 17:27:08 2022] GET /identity/v3/users/cc2669e1aa494dbcb7164d2e5e46378c => generated 247 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:27:08.655254 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1636f2f5-9540-483a-b26c-a0b5798daefe None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:08.655627 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1636f2f5-9540-483a-b26c-a0b5798daefe None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:08.655803 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1636f2f5-9540-483a-b26c-a0b5798daefe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:08.679769 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 86/174] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Aug 24 17:27:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 509 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 17:27:08.680969 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cac1a61-0ba8-4a64-8721-95363c97fd1a None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:08.681646 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-1636f2f5-9540-483a-b26c-a0b5798daefe None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:27:08.682886 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-1636f2f5-9540-483a-b26c-a0b5798daefe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:27:08.756707 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:08.769466 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 89/175] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:27:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:27:08.770855 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9af83f9-de47-476a-9e6e-6810ec47f515 None None] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:08.771142 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9af83f9-de47-476a-9e6e-6810ec47f515 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:08.771415 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9af83f9-de47-476a-9e6e-6810ec47f515 None None] PATH_INFO: `/` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:08.772712 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 90/176] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:27:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:27:08.776774 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9af83f9-de47-476a-9e6e-6810ec47f515 None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:08.812707 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cac1a61-0ba8-4a64-8721-95363c97fd1a None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:08.846222 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b720d2a7-84d6-4b5f-9557-659864f59a31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:08.846850 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b720d2a7-84d6-4b5f-9557-659864f59a31 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:08.847193 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b720d2a7-84d6-4b5f-9557-659864f59a31 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:08.847484 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b720d2a7-84d6-4b5f-9557-659864f59a31 None admin] PATH_INFO: `/v3/projects/cc1182fc1781499884cebfaacd1ef5a0` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:08.848596 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b720d2a7-84d6-4b5f-9557-659864f59a31 None admin] RBAC: Authorizing `identity:get_project(project_id=cc1182fc1781499884cebfaacd1ef5a0)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:08.850044 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b720d2a7-84d6-4b5f-9557-659864f59a31 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:08.851813 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 87/177] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Aug 24 17:27:08 2022] GET /identity/v3/projects/cc1182fc1781499884cebfaacd1ef5a0 => generated 298 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:27:08.852416 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b720d2a7-84d6-4b5f-9557-659864f59a31 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:08.890427 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:08.891328 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9af83f9-de47-476a-9e6e-6810ec47f515 None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:08.927471 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c5076e-fc27-4867-8bae-486bdf0ace2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:08.927701 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c5076e-fc27-4867-8bae-486bdf0ace2d None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:08.927955 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c5076e-fc27-4867-8bae-486bdf0ace2d None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:08.928226 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c5076e-fc27-4867-8bae-486bdf0ace2d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:08.957367 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b720d2a7-84d6-4b5f-9557-659864f59a31 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:08.987637 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:08.994539 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15041e79-53ec-42df-80c3-35ada0041983 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:08.995672 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15041e79-53ec-42df-80c3-35ada0041983 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:08.995957 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15041e79-53ec-42df-80c3-35ada0041983 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:08.996232 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15041e79-53ec-42df-80c3-35ada0041983 None admin] PATH_INFO: `/v3/projects/e615e00f222d4a298a1f7ceea74eb618` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:08.997393 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15041e79-53ec-42df-80c3-35ada0041983 None admin] RBAC: Authorizing `identity:get_project(project_id=e615e00f222d4a298a1f7ceea74eb618)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:08.998864 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15041e79-53ec-42df-80c3-35ada0041983 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:08.999935 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c5076e-fc27-4867-8bae-486bdf0ace2d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:09.000491 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 88/178] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Aug 24 17:27:08 2022] GET /identity/v3/projects/e615e00f222d4a298a1f7ceea74eb618 => generated 312 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:27:09.001635 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8bc6e2-9511-42dc-9422-5b1bda181fcd None None] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:09.001918 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8bc6e2-9511-42dc-9422-5b1bda181fcd None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:09.002192 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8bc6e2-9511-42dc-9422-5b1bda181fcd None None] PATH_INFO: `/` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:09.002462 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c5076e-fc27-4867-8bae-486bdf0ace2d None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:09.003686 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 89/179] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:27:09 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:27:09.007999 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfd063c-692b-49c4-8ded-1362c9348d29 None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:09.007999 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfd063c-692b-49c4-8ded-1362c9348d29 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:09.007999 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfd063c-692b-49c4-8ded-1362c9348d29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:09.023064 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-4bfd063c-692b-49c4-8ded-1362c9348d29 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:27:09.024331 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-4bfd063c-692b-49c4-8ded-1362c9348d29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:27:09.089314 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 90/180] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:27:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:27:09.090680 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa186c93-8050-4b21-9830-a0db974e7e96 None None] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:09.090970 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa186c93-8050-4b21-9830-a0db974e7e96 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:09.091303 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa186c93-8050-4b21-9830-a0db974e7e96 None None] PATH_INFO: `/` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:09.092630 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 91/181] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:27:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:27:09.093329 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa186c93-8050-4b21-9830-a0db974e7e96 None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:09.116133 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 91/182] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Aug 24 17:27:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 17:27:09.117265 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c5076e-fc27-4867-8bae-486bdf0ace2d None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:09.131405 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:09.200116 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa186c93-8050-4b21-9830-a0db974e7e96 None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:09.222754 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:09.235000 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062fd981-cc54-46a7-9ae2-a2dd18dcab08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:09.235274 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c5076e-fc27-4867-8bae-486bdf0ace2d None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:09.235613 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062fd981-cc54-46a7-9ae2-a2dd18dcab08 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:09.235893 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062fd981-cc54-46a7-9ae2-a2dd18dcab08 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:09.236164 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062fd981-cc54-46a7-9ae2-a2dd18dcab08 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:09.236922 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062fd981-cc54-46a7-9ae2-a2dd18dcab08 None admin] RBAC: Adding query filter params (role.id=148f135e4a134697a032be04bc9faa4c, scope.project.id=cc1182fc1781499884cebfaacd1ef5a0, user.id=1f7a270ae41e45eab15c82f41826ca94) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:27:09.237231 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062fd981-cc54-46a7-9ae2-a2dd18dcab08 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:09.238670 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062fd981-cc54-46a7-9ae2-a2dd18dcab08 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:09.256816 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 92/183] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Aug 24 17:27:09 2022] GET /identity/v3/role_assignments?user.id=1f7a270ae41e45eab15c82f41826ca94&scope.project.id=cc1182fc1781499884cebfaacd1ef5a0&role.id=148f135e4a134697a032be04bc9faa4c => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:27:09.256816 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da301ada-1558-4236-97f0-aa3ec2b56071 None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:09.256816 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da301ada-1558-4236-97f0-aa3ec2b56071 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:09.256816 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da301ada-1558-4236-97f0-aa3ec2b56071 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:09.267935 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-da301ada-1558-4236-97f0-aa3ec2b56071 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:27:09.269630 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-da301ada-1558-4236-97f0-aa3ec2b56071 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:27:09.289711 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f595401-d5a6-4ea7-9c45-e6b9c907b196 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:09.290330 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f595401-d5a6-4ea7-9c45-e6b9c907b196 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:09.290609 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f595401-d5a6-4ea7-9c45-e6b9c907b196 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:09.290882 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f595401-d5a6-4ea7-9c45-e6b9c907b196 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:09.292270 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f595401-d5a6-4ea7-9c45-e6b9c907b196 None admin] RBAC: Adding query filter params (role.id=02476a5b82eb4b188c18a5d706a3ef68, scope.project.id=e615e00f222d4a298a1f7ceea74eb618, user.id=1f7a270ae41e45eab15c82f41826ca94) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:27:09.292614 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f595401-d5a6-4ea7-9c45-e6b9c907b196 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:09.294111 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f595401-d5a6-4ea7-9c45-e6b9c907b196 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:09.306343 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 92/184] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Aug 24 17:27:09 2022] GET /identity/v3/role_assignments?user.id=1f7a270ae41e45eab15c82f41826ca94&scope.project.id=e615e00f222d4a298a1f7ceea74eb618&role.id=02476a5b82eb4b188c18a5d706a3ef68 => generated 608 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:27:09.307576 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f595401-d5a6-4ea7-9c45-e6b9c907b196 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:09.364284 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 93/185] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:27:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:27:09.367144 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e033fe-e9ed-49ab-ae21-8d94e33ca344 None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:09.367424 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e033fe-e9ed-49ab-ae21-8d94e33ca344 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:09.367696 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e033fe-e9ed-49ab-ae21-8d94e33ca344 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:09.400239 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-d0e033fe-e9ed-49ab-ae21-8d94e33ca344 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:27:09.401566 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-d0e033fe-e9ed-49ab-ae21-8d94e33ca344 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:27:09.434561 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:09.491624 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f595401-d5a6-4ea7-9c45-e6b9c907b196 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:09.491804 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 94/186] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:27:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:27:09.492524 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e033fe-e9ed-49ab-ae21-8d94e33ca344 None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:09.527935 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a56426-c26e-4326-8df3-0b79167a3f85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:09.528984 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a56426-c26e-4326-8df3-0b79167a3f85 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:09.529491 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a56426-c26e-4326-8df3-0b79167a3f85 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:09.529980 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a56426-c26e-4326-8df3-0b79167a3f85 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:09.540679 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a56426-c26e-4326-8df3-0b79167a3f85 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:27:09.548496 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application [None req-c5a56426-c26e-4326-8df3-0b79167a3f85 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:27:09.548496 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:27:09.548496 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:27:09.548496 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:27:09.548496 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:27:09.548496 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:27:09.548496 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:27:09.548496 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:27:09.548496 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:27:09.548496 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:27:09.548496 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:27:09.548496 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:27:09.548496 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:27:09.548496 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:27:09.548496 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:27:09.548496 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:27:09.549466 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:27:09.549466 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:27:09.549466 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:27:09.549466 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:27:09.549466 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:27:09.549466 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:27:09.549466 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:27:09.549466 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:27:09.549466 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:27:09.549466 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:27:09.549466 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:27:09.549466 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application with Lock( Aug 24 17:27:09.549466 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:27:09.549466 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:27:09.549466 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:27:09.549466 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:27:09.550262 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:27:09.550262 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:27:09.550262 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:27:09.550262 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:27:09.550262 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:27:09.550262 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:27:09.550262 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:27:09.550262 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:27:09.550262 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:27:09.550262 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:27:09.550262 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:27:09.550262 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Aug 24 17:27:09.552582 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 93/187] 10.222.0.45 () {56 vars in 1096 bytes} [Wed Aug 24 17:27:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:27:09.553561 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a56426-c26e-4326-8df3-0b79167a3f85 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:09.583084 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:09.604486 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e033fe-e9ed-49ab-ae21-8d94e33ca344 None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:09.644807 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25c2e51-2df4-4b91-aa72-271042868469 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:09.645444 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25c2e51-2df4-4b91-aa72-271042868469 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:09.645998 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25c2e51-2df4-4b91-aa72-271042868469 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:09.645998 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25c2e51-2df4-4b91-aa72-271042868469 None admin] PATH_INFO: `/v3/projects/cc1182fc1781499884cebfaacd1ef5a0` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:09.647099 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25c2e51-2df4-4b91-aa72-271042868469 None admin] RBAC: Authorizing `identity:get_project(project_id=cc1182fc1781499884cebfaacd1ef5a0)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:09.648578 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25c2e51-2df4-4b91-aa72-271042868469 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:09.651668 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 95/188] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Aug 24 17:27:09 2022] GET /identity/v3/projects/cc1182fc1781499884cebfaacd1ef5a0 => generated 298 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:27:09.652466 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25c2e51-2df4-4b91-aa72-271042868469 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:09.665616 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a56426-c26e-4326-8df3-0b79167a3f85 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:09.680965 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:09.706887 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f903783-0bad-43b5-9d2e-2506055e6dc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:09.707908 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f903783-0bad-43b5-9d2e-2506055e6dc5 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:09.708401 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f903783-0bad-43b5-9d2e-2506055e6dc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:09.708922 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f903783-0bad-43b5-9d2e-2506055e6dc5 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:09.720165 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f903783-0bad-43b5-9d2e-2506055e6dc5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:27:09.725808 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application [None req-0f903783-0bad-43b5-9d2e-2506055e6dc5 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 17:27:09.725808 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:27:09.725808 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:27:09.725808 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:27:09.725808 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:27:09.725808 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:27:09.725808 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:27:09.725808 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:27:09.725808 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:27:09.725808 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:27:09.725808 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:27:09.725808 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:27:09.725808 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:27:09.725808 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:27:09.725808 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:27:09.725808 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:27:09.726775 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:27:09.726775 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:27:09.726775 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:27:09.726775 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:27:09.726775 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:27:09.726775 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:27:09.726775 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:27:09.726775 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:27:09.726775 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:27:09.726775 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:27:09.726775 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:27:09.726775 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application with Lock( Aug 24 17:27:09.726775 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:27:09.726775 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:27:09.726775 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:27:09.726775 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:27:09.727896 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:27:09.727896 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:27:09.727896 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:27:09.727896 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:27:09.727896 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:27:09.727896 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:27:09.727896 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:27:09.727896 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:27:09.727896 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:27:09.727896 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:27:09.727896 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 17:27:09.727896 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Aug 24 17:27:09.727896 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 94/189] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Aug 24 17:27:09 2022] GET /identity/v3/roles/reader => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:27:09.728640 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f903783-0bad-43b5-9d2e-2506055e6dc5 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:09.768041 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25c2e51-2df4-4b91-aa72-271042868469 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:09.809718 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54eda537-5ba8-45fa-acd3-bb0b915071d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:09.810344 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54eda537-5ba8-45fa-acd3-bb0b915071d0 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:09.810718 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54eda537-5ba8-45fa-acd3-bb0b915071d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:09.811091 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54eda537-5ba8-45fa-acd3-bb0b915071d0 None admin] PATH_INFO: `/v3/roles` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:09.811771 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54eda537-5ba8-45fa-acd3-bb0b915071d0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:27:09.812075 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54eda537-5ba8-45fa-acd3-bb0b915071d0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:09.813481 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54eda537-5ba8-45fa-acd3-bb0b915071d0 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:09.817113 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:09.822358 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 96/190] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Aug 24 17:27:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:27:09.823144 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54eda537-5ba8-45fa-acd3-bb0b915071d0 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:09.847958 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f903783-0bad-43b5-9d2e-2506055e6dc5 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:09.880603 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d378a013-8c42-4822-b50e-2328dc7d1978 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:09.881212 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d378a013-8c42-4822-b50e-2328dc7d1978 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:09.881492 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d378a013-8c42-4822-b50e-2328dc7d1978 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:09.881763 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d378a013-8c42-4822-b50e-2328dc7d1978 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:09.882600 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d378a013-8c42-4822-b50e-2328dc7d1978 None admin] RBAC: Adding query filter params (role.id=e411ad9eb96e4e8d85b59b5b700ada7b, scope.project.id=cc1182fc1781499884cebfaacd1ef5a0, user.id=cc2669e1aa494dbcb7164d2e5e46378c) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:27:09.882787 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d378a013-8c42-4822-b50e-2328dc7d1978 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:09.884232 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d378a013-8c42-4822-b50e-2328dc7d1978 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:09.890612 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 95/191] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Aug 24 17:27:09 2022] GET /identity/v3/role_assignments?user.id=cc2669e1aa494dbcb7164d2e5e46378c&scope.project.id=cc1182fc1781499884cebfaacd1ef5a0&role.id=e411ad9eb96e4e8d85b59b5b700ada7b => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:27:09.891261 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d378a013-8c42-4822-b50e-2328dc7d1978 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:09.910750 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:09.925140 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54eda537-5ba8-45fa-acd3-bb0b915071d0 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:09.963671 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1869563-2022-4c3e-a79d-839bb6ab5a44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:09.964325 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1869563-2022-4c3e-a79d-839bb6ab5a44 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:09.964630 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1869563-2022-4c3e-a79d-839bb6ab5a44 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:09.964905 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1869563-2022-4c3e-a79d-839bb6ab5a44 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:09.966078 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1869563-2022-4c3e-a79d-839bb6ab5a44 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:09.967557 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1869563-2022-4c3e-a79d-839bb6ab5a44 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:09.969398 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 97/192] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 17:27:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:27:09.970209 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1869563-2022-4c3e-a79d-839bb6ab5a44 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:10.016628 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d378a013-8c42-4822-b50e-2328dc7d1978 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:10.057700 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8572503-9153-4e64-8e4b-6e27b9c111d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:10.058245 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8572503-9153-4e64-8e4b-6e27b9c111d8 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:10.058521 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8572503-9153-4e64-8e4b-6e27b9c111d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:10.058793 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8572503-9153-4e64-8e4b-6e27b9c111d8 None admin] PATH_INFO: `/v3/roles` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:10.059834 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8572503-9153-4e64-8e4b-6e27b9c111d8 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:27:10.059834 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8572503-9153-4e64-8e4b-6e27b9c111d8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:10.061209 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8572503-9153-4e64-8e4b-6e27b9c111d8 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:10.069805 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:10.075098 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 96/193] 10.222.0.45 () {56 vars in 1101 bytes} [Wed Aug 24 17:27:09 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:27:10.076598 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8572503-9153-4e64-8e4b-6e27b9c111d8 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:10.100834 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1869563-2022-4c3e-a79d-839bb6ab5a44 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:10.137029 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a12243-166e-4029-a33c-bfcd20f5e440 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:10.137674 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a12243-166e-4029-a33c-bfcd20f5e440 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:10.137957 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a12243-166e-4029-a33c-bfcd20f5e440 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:10.138657 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a12243-166e-4029-a33c-bfcd20f5e440 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:10.149044 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a12243-166e-4029-a33c-bfcd20f5e440 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:27:10.154262 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application [None req-56a12243-166e-4029-a33c-bfcd20f5e440 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 17:27:10.154262 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:27:10.154262 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:27:10.154262 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:27:10.154262 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:27:10.154262 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:27:10.154262 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:27:10.154262 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:27:10.154262 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:27:10.154262 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:27:10.154262 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:27:10.154262 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:27:10.154262 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:27:10.154262 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:27:10.154262 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:27:10.154262 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:27:10.155118 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:27:10.155118 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:27:10.155118 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:27:10.155118 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:27:10.155118 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:27:10.155118 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:27:10.155118 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:27:10.155118 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:27:10.155118 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:27:10.155118 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:27:10.155118 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:27:10.155118 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application with Lock( Aug 24 17:27:10.155118 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:27:10.155118 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:27:10.155118 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:27:10.155118 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:27:10.156310 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:27:10.156310 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:27:10.156310 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:27:10.156310 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:27:10.156310 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:27:10.156310 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:27:10.156310 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:27:10.156310 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:27:10.156310 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:27:10.156310 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:27:10.156310 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 17:27:10.156310 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Aug 24 17:27:10.156310 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 98/194] 10.222.0.45 () {56 vars in 1114 bytes} [Wed Aug 24 17:27:09 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:27:10.157206 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a12243-166e-4029-a33c-bfcd20f5e440 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:10.174514 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:10.185218 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8572503-9153-4e64-8e4b-6e27b9c111d8 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:10.217706 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df43b787-7c3d-48a6-b192-e18f3b4bf34c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:10.218363 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df43b787-7c3d-48a6-b192-e18f3b4bf34c None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:10.218638 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df43b787-7c3d-48a6-b192-e18f3b4bf34c None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:10.218905 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df43b787-7c3d-48a6-b192-e18f3b4bf34c None admin] PATH_INFO: `/v3/users/2978a18a2c1b4832aeb028089640fd0d` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:10.232253 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df43b787-7c3d-48a6-b192-e18f3b4bf34c None admin] RBAC: Authorizing `identity:get_user(user_id=2978a18a2c1b4832aeb028089640fd0d)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:10.233747 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df43b787-7c3d-48a6-b192-e18f3b4bf34c None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:10.235335 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 97/195] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Aug 24 17:27:10 2022] GET /identity/v3/users/2978a18a2c1b4832aeb028089640fd0d => generated 283 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:27:10.236084 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df43b787-7c3d-48a6-b192-e18f3b4bf34c None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:10.261398 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a12243-166e-4029-a33c-bfcd20f5e440 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:10.291896 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:10.294222 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cd0a61-a1df-4613-afc2-f309b2cb6918 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:10.295135 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cd0a61-a1df-4613-afc2-f309b2cb6918 None admin] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:10.295420 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cd0a61-a1df-4613-afc2-f309b2cb6918 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:10.295684 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cd0a61-a1df-4613-afc2-f309b2cb6918 None admin] PATH_INFO: `/v3/users` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:10.296359 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cd0a61-a1df-4613-afc2-f309b2cb6918 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:10.297847 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cd0a61-a1df-4613-afc2-f309b2cb6918 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:10.318084 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 99/196] 10.222.0.45 () {60 vars in 1132 bytes} [Wed Aug 24 17:27:10 2022] POST /identity/v3/users => generated 297 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:27:10.318839 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cd0a61-a1df-4613-afc2-f309b2cb6918 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:10.352019 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df43b787-7c3d-48a6-b192-e18f3b4bf34c None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:10.391737 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ec8e11-86e9-4d85-be9d-13eece373518 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:10.392391 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ec8e11-86e9-4d85-be9d-13eece373518 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:10.392700 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ec8e11-86e9-4d85-be9d-13eece373518 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:10.392977 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ec8e11-86e9-4d85-be9d-13eece373518 None admin] PATH_INFO: `/v3/users/9bac15d3c1cc40f9bbfa687de74340b4` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:10.394355 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ec8e11-86e9-4d85-be9d-13eece373518 None admin] RBAC: Authorizing `identity:get_user(user_id=9bac15d3c1cc40f9bbfa687de74340b4)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:10.395853 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ec8e11-86e9-4d85-be9d-13eece373518 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:10.397566 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 98/197] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Aug 24 17:27:10 2022] GET /identity/v3/users/9bac15d3c1cc40f9bbfa687de74340b4 => generated 289 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:27:10.398358 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ec8e11-86e9-4d85-be9d-13eece373518 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:10.407765 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:10.444275 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cd0a61-a1df-4613-afc2-f309b2cb6918 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:10.483037 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b530d77a-7827-4c19-a463-74cbfa4a86d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:10.484083 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b530d77a-7827-4c19-a463-74cbfa4a86d3 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:10.484937 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b530d77a-7827-4c19-a463-74cbfa4a86d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:10.485219 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b530d77a-7827-4c19-a463-74cbfa4a86d3 None admin] PATH_INFO: `/v3/roles` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:10.485917 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b530d77a-7827-4c19-a463-74cbfa4a86d3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:27:10.486219 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b530d77a-7827-4c19-a463-74cbfa4a86d3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:10.487613 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b530d77a-7827-4c19-a463-74cbfa4a86d3 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:10.496058 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 100/198] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Aug 24 17:27:10 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:27:10.496958 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b530d77a-7827-4c19-a463-74cbfa4a86d3 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:10.520009 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ec8e11-86e9-4d85-be9d-13eece373518 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:10.551446 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:10.566060 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fa258b-25cf-4b94-9085-4a10288f8fba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:10.567138 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fa258b-25cf-4b94-9085-4a10288f8fba None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:10.567621 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fa258b-25cf-4b94-9085-4a10288f8fba None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:10.568310 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fa258b-25cf-4b94-9085-4a10288f8fba None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:10.620826 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b530d77a-7827-4c19-a463-74cbfa4a86d3 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:10.657393 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbcd318-d4f4-4418-a1f6-e472fff42146 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:10.658011 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbcd318-d4f4-4418-a1f6-e472fff42146 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:10.658293 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbcd318-d4f4-4418-a1f6-e472fff42146 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:10.658577 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbcd318-d4f4-4418-a1f6-e472fff42146 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:10.661341 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fa258b-25cf-4b94-9085-4a10288f8fba None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:10.663018 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:10.665571 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fa258b-25cf-4b94-9085-4a10288f8fba None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:10.734784 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbcd318-d4f4-4418-a1f6-e472fff42146 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:10.736718 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbcd318-d4f4-4418-a1f6-e472fff42146 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:10.804035 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 99/199] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Aug 24 17:27:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 17:27:10.804741 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fa258b-25cf-4b94-9085-4a10288f8fba None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:10.838559 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:10.871742 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 101/200] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Aug 24 17:27:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 17:27:10.876955 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbcd318-d4f4-4418-a1f6-e472fff42146 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:10.909961 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:10.931750 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fa258b-25cf-4b94-9085-4a10288f8fba None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:10.981032 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbcd318-d4f4-4418-a1f6-e472fff42146 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:10.981693 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2ad760-12ac-4750-a52a-6cb068a2c05e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:10.982339 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2ad760-12ac-4750-a52a-6cb068a2c05e None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:10.982871 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2ad760-12ac-4750-a52a-6cb068a2c05e None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:10.983145 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2ad760-12ac-4750-a52a-6cb068a2c05e None admin] PATH_INFO: `/v3/users/2978a18a2c1b4832aeb028089640fd0d` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:10.985213 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2ad760-12ac-4750-a52a-6cb068a2c05e None admin] RBAC: Authorizing `identity:get_user(user_id=2978a18a2c1b4832aeb028089640fd0d)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:10.986677 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2ad760-12ac-4750-a52a-6cb068a2c05e None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:10.988277 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 100/201] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Aug 24 17:27:10 2022] GET /identity/v3/users/2978a18a2c1b4832aeb028089640fd0d => generated 283 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:27:10.989610 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2ad760-12ac-4750-a52a-6cb068a2c05e None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:11.019231 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd8dc7a-fb40-412f-9f06-053d65bb3823 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:11.020001 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd8dc7a-fb40-412f-9f06-053d65bb3823 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:11.020351 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd8dc7a-fb40-412f-9f06-053d65bb3823 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:11.020718 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd8dc7a-fb40-412f-9f06-053d65bb3823 None admin] PATH_INFO: `/v3/projects/3a29643b449b4fc1aa71802d8db7c0d2` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:11.021931 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd8dc7a-fb40-412f-9f06-053d65bb3823 None admin] RBAC: Authorizing `identity:get_project(project_id=3a29643b449b4fc1aa71802d8db7c0d2)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:11.023450 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd8dc7a-fb40-412f-9f06-053d65bb3823 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:11.025228 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 102/202] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Aug 24 17:27:10 2022] GET /identity/v3/projects/3a29643b449b4fc1aa71802d8db7c0d2 => generated 302 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:27:11.026690 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd8dc7a-fb40-412f-9f06-053d65bb3823 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:11.107274 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:11.115365 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2ad760-12ac-4750-a52a-6cb068a2c05e None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:11.148131 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd8dc7a-fb40-412f-9f06-053d65bb3823 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:11.149899 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:11.154002 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3ca260-a6bf-4328-98ca-ce22132c0f0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:11.154725 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3ca260-a6bf-4328-98ca-ce22132c0f0b None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:11.155077 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3ca260-a6bf-4328-98ca-ce22132c0f0b None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:11.155421 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3ca260-a6bf-4328-98ca-ce22132c0f0b None admin] PATH_INFO: `/v3/projects/cc1182fc1781499884cebfaacd1ef5a0` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:11.156695 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3ca260-a6bf-4328-98ca-ce22132c0f0b None admin] RBAC: Authorizing `identity:get_project(project_id=cc1182fc1781499884cebfaacd1ef5a0)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:11.158338 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3ca260-a6bf-4328-98ca-ce22132c0f0b None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:11.160523 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 101/203] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Aug 24 17:27:10 2022] GET /identity/v3/projects/cc1182fc1781499884cebfaacd1ef5a0 => generated 298 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:27:11.161641 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3ca260-a6bf-4328-98ca-ce22132c0f0b None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:11.193605 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31b552f-7ab7-4817-a887-5f1e391d7eb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:11.194486 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31b552f-7ab7-4817-a887-5f1e391d7eb1 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:11.194765 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31b552f-7ab7-4817-a887-5f1e391d7eb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:11.195037 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31b552f-7ab7-4817-a887-5f1e391d7eb1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:11.272158 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3ca260-a6bf-4328-98ca-ce22132c0f0b None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:11.291640 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31b552f-7ab7-4817-a887-5f1e391d7eb1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:11.293649 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31b552f-7ab7-4817-a887-5f1e391d7eb1 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:11.308862 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023c267e-29e8-4613-a5f2-3016a27f6cb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:11.309706 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023c267e-29e8-4613-a5f2-3016a27f6cb0 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:11.310082 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023c267e-29e8-4613-a5f2-3016a27f6cb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:11.310449 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023c267e-29e8-4613-a5f2-3016a27f6cb0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:11.311461 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023c267e-29e8-4613-a5f2-3016a27f6cb0 None admin] RBAC: Adding query filter params (role.id=e411ad9eb96e4e8d85b59b5b700ada7b, scope.project.id=3a29643b449b4fc1aa71802d8db7c0d2, user.id=2978a18a2c1b4832aeb028089640fd0d) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:27:11.311974 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023c267e-29e8-4613-a5f2-3016a27f6cb0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:11.313633 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023c267e-29e8-4613-a5f2-3016a27f6cb0 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:11.320010 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 102/204] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Aug 24 17:27:11 2022] GET /identity/v3/role_assignments?user.id=2978a18a2c1b4832aeb028089640fd0d&scope.project.id=3a29643b449b4fc1aa71802d8db7c0d2&role.id=e411ad9eb96e4e8d85b59b5b700ada7b => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:27:11.320995 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023c267e-29e8-4613-a5f2-3016a27f6cb0 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:11.356063 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:11.470318 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 103/205] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Aug 24 17:27:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 444 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 17:27:11.475908 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31b552f-7ab7-4817-a887-5f1e391d7eb1 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:11.478320 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:11.501637 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023c267e-29e8-4613-a5f2-3016a27f6cb0 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:11.549080 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62e13d2-20d9-4110-95b5-7682e1f12062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:11.549692 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62e13d2-20d9-4110-95b5-7682e1f12062 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:11.549965 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62e13d2-20d9-4110-95b5-7682e1f12062 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:11.550233 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62e13d2-20d9-4110-95b5-7682e1f12062 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:11.550960 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62e13d2-20d9-4110-95b5-7682e1f12062 None admin] RBAC: Adding query filter params (role.id=645db6d227604357b1949889630755ff, scope.project.id=cc1182fc1781499884cebfaacd1ef5a0, user.id=9bac15d3c1cc40f9bbfa687de74340b4) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:27:11.551263 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62e13d2-20d9-4110-95b5-7682e1f12062 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:11.552745 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62e13d2-20d9-4110-95b5-7682e1f12062 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:11.565276 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 103/206] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Aug 24 17:27:11 2022] GET /identity/v3/role_assignments?user.id=9bac15d3c1cc40f9bbfa687de74340b4&scope.project.id=cc1182fc1781499884cebfaacd1ef5a0&role.id=645db6d227604357b1949889630755ff => generated 608 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:27:11.669782 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31b552f-7ab7-4817-a887-5f1e391d7eb1 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:11.706705 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992ac125-ce3c-4cb8-84d0-72dfcd06b864 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:11.707421 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992ac125-ce3c-4cb8-84d0-72dfcd06b864 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:11.707704 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992ac125-ce3c-4cb8-84d0-72dfcd06b864 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:11.707972 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992ac125-ce3c-4cb8-84d0-72dfcd06b864 None admin] PATH_INFO: `/v3/projects/3a29643b449b4fc1aa71802d8db7c0d2` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:11.709159 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992ac125-ce3c-4cb8-84d0-72dfcd06b864 None admin] RBAC: Authorizing `identity:get_project(project_id=3a29643b449b4fc1aa71802d8db7c0d2)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:11.710629 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992ac125-ce3c-4cb8-84d0-72dfcd06b864 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:11.712676 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 104/207] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Aug 24 17:27:11 2022] GET /identity/v3/projects/3a29643b449b4fc1aa71802d8db7c0d2 => generated 302 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:27:11.721777 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62e13d2-20d9-4110-95b5-7682e1f12062 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:11.851470 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:11.864272 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62e13d2-20d9-4110-95b5-7682e1f12062 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:11.903290 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71bb31c-6287-4f31-8e07-25e3cde4504f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:11.903900 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71bb31c-6287-4f31-8e07-25e3cde4504f None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:11.904176 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71bb31c-6287-4f31-8e07-25e3cde4504f None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:11.904448 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71bb31c-6287-4f31-8e07-25e3cde4504f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:11.905310 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71bb31c-6287-4f31-8e07-25e3cde4504f None admin] RBAC: Adding query filter params (role.id=148f135e4a134697a032be04bc9faa4c, scope.project.id=3a29643b449b4fc1aa71802d8db7c0d2, user.id=2978a18a2c1b4832aeb028089640fd0d) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:27:11.905623 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71bb31c-6287-4f31-8e07-25e3cde4504f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:11.907084 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71bb31c-6287-4f31-8e07-25e3cde4504f None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:11.913196 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 104/208] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Aug 24 17:27:11 2022] GET /identity/v3/role_assignments?user.id=2978a18a2c1b4832aeb028089640fd0d&scope.project.id=3a29643b449b4fc1aa71802d8db7c0d2&role.id=148f135e4a134697a032be04bc9faa4c => generated 259 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:27:12.510733 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d075fab1-de3d-4213-901b-2a681c6e4662 None None] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:12.511245 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d075fab1-de3d-4213-901b-2a681c6e4662 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:12.511349 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d075fab1-de3d-4213-901b-2a681c6e4662 None None] PATH_INFO: `/` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:12.512970 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 105/209] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:27:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:27:12.518304 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a0a8db-43d0-4180-bf08-38e308a5abf0 None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:12.518691 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a0a8db-43d0-4180-bf08-38e308a5abf0 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:12.518852 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a0a8db-43d0-4180-bf08-38e308a5abf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:12.538742 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-c5a0a8db-43d0-4180-bf08-38e308a5abf0 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:27:12.539977 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-c5a0a8db-43d0-4180-bf08-38e308a5abf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:27:12.545018 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1c2fc8-5a30-41a9-a928-c983fd20b1f7 None None] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:12.545329 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1c2fc8-5a30-41a9-a928-c983fd20b1f7 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:12.545568 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1c2fc8-5a30-41a9-a928-c983fd20b1f7 None None] PATH_INFO: `/` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:12.547295 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 106/210] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:27:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:27:12.553095 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7787e40-912e-40b8-a153-8af9da625d82 None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:12.553391 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7787e40-912e-40b8-a153-8af9da625d82 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:12.553646 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7787e40-912e-40b8-a153-8af9da625d82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:12.575717 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-f7787e40-912e-40b8-a153-8af9da625d82 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:27:12.576948 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-f7787e40-912e-40b8-a153-8af9da625d82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:27:12.610920 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 105/211] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:27:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:27:12.651028 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:12.656195 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 107/212] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:27:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 17:27:12.881166 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a0a8db-43d0-4180-bf08-38e308a5abf0 None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:12.945038 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7787e40-912e-40b8-a153-8af9da625d82 None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:12.985746 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a0a8db-43d0-4180-bf08-38e308a5abf0 None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:13.001316 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:13.029740 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd3789a-f0e6-488e-b914-c1df335d15c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:13.031126 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd3789a-f0e6-488e-b914-c1df335d15c9 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:13.031879 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd3789a-f0e6-488e-b914-c1df335d15c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:13.032349 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd3789a-f0e6-488e-b914-c1df335d15c9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:13.033997 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd3789a-f0e6-488e-b914-c1df335d15c9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:13.036070 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd3789a-f0e6-488e-b914-c1df335d15c9 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:13.039070 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 106/213] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 17:27:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:27:13.048986 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd3789a-f0e6-488e-b914-c1df335d15c9 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:13.054279 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7787e40-912e-40b8-a153-8af9da625d82 None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:13.091171 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172d24d2-9f78-461b-aa1c-1c5833ccf2df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:13.091899 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172d24d2-9f78-461b-aa1c-1c5833ccf2df None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:13.092242 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172d24d2-9f78-461b-aa1c-1c5833ccf2df None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:13.092612 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172d24d2-9f78-461b-aa1c-1c5833ccf2df None admin] PATH_INFO: `/v3/roles/member` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:13.103636 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172d24d2-9f78-461b-aa1c-1c5833ccf2df None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:27:13.108992 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application [None req-172d24d2-9f78-461b-aa1c-1c5833ccf2df None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 17:27:13.108992 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:27:13.108992 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:27:13.108992 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:27:13.108992 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:27:13.108992 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:27:13.108992 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:27:13.108992 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:27:13.108992 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:27:13.108992 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:27:13.108992 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:27:13.108992 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:27:13.108992 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:27:13.108992 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:27:13.108992 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:27:13.108992 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:27:13.109852 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:27:13.109852 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:27:13.109852 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:27:13.109852 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:27:13.109852 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:27:13.109852 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:27:13.109852 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:27:13.109852 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:27:13.109852 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:27:13.109852 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:27:13.109852 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:27:13.109852 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application with Lock( Aug 24 17:27:13.109852 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:27:13.109852 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:27:13.109852 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:27:13.109852 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:27:13.110659 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:27:13.110659 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:27:13.110659 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:27:13.110659 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:27:13.110659 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:27:13.110659 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:27:13.110659 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:27:13.110659 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:27:13.110659 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:27:13.110659 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:27:13.110659 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 17:27:13.110659 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Aug 24 17:27:13.110659 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 108/214] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Aug 24 17:27:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 17:27:13.116057 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172d24d2-9f78-461b-aa1c-1c5833ccf2df None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:13.169060 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd3789a-f0e6-488e-b914-c1df335d15c9 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:13.178707 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:13.208192 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dca6806-a790-40a3-a5b6-11b47c0c84e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:13.208924 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dca6806-a790-40a3-a5b6-11b47c0c84e9 None admin] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:13.209269 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dca6806-a790-40a3-a5b6-11b47c0c84e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:13.209604 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dca6806-a790-40a3-a5b6-11b47c0c84e9 None admin] PATH_INFO: `/v3/users` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:13.210345 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dca6806-a790-40a3-a5b6-11b47c0c84e9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:13.212044 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dca6806-a790-40a3-a5b6-11b47c0c84e9 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:13.232445 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172d24d2-9f78-461b-aa1c-1c5833ccf2df None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:13.238039 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 107/215] 10.222.0.45 () {60 vars in 1132 bytes} [Wed Aug 24 17:27:13 2022] POST /identity/v3/users => generated 297 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:27:13.275076 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf68403-4d42-4bf4-b661-16b295c811f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:13.275749 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf68403-4d42-4bf4-b661-16b295c811f7 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:13.276033 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf68403-4d42-4bf4-b661-16b295c811f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:13.276359 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf68403-4d42-4bf4-b661-16b295c811f7 None admin] PATH_INFO: `/v3/roles` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:13.277135 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf68403-4d42-4bf4-b661-16b295c811f7 None admin] RBAC: Adding query filter params (name=member) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:27:13.277443 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf68403-4d42-4bf4-b661-16b295c811f7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:13.278875 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf68403-4d42-4bf4-b661-16b295c811f7 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:13.288761 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 109/216] 10.222.0.45 () {56 vars in 1101 bytes} [Wed Aug 24 17:27:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:27:13.295116 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dca6806-a790-40a3-a5b6-11b47c0c84e9 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:13.308592 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:13.412456 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ffdfda-dff9-44cb-8e20-4b456683324b None None] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:13.412670 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dca6806-a790-40a3-a5b6-11b47c0c84e9 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:13.412884 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ffdfda-dff9-44cb-8e20-4b456683324b None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:13.413053 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ffdfda-dff9-44cb-8e20-4b456683324b None None] PATH_INFO: `/` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:13.414595 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 110/217] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:27:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 17:27:13.420077 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca1ef51-a426-4f68-858f-0e56cb69c470 None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:13.420365 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca1ef51-a426-4f68-858f-0e56cb69c470 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:13.420718 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca1ef51-a426-4f68-858f-0e56cb69c470 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:13.445442 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-fca1ef51-a426-4f68-858f-0e56cb69c470 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:27:13.446779 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-fca1ef51-a426-4f68-858f-0e56cb69c470 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:27:13.458279 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d8ae26-f040-4098-b44d-42b2c56802df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:13.459113 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d8ae26-f040-4098-b44d-42b2c56802df None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:13.459306 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d8ae26-f040-4098-b44d-42b2c56802df None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:13.459570 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d8ae26-f040-4098-b44d-42b2c56802df None admin] PATH_INFO: `/v3/users/d5e7e3bd287e431eba21ba5fa1bb0587` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:13.480360 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d8ae26-f040-4098-b44d-42b2c56802df None admin] RBAC: Authorizing `identity:get_user(user_id=d5e7e3bd287e431eba21ba5fa1bb0587)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:13.481871 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d8ae26-f040-4098-b44d-42b2c56802df None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:13.483758 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 108/218] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Aug 24 17:27:13 2022] GET /identity/v3/users/d5e7e3bd287e431eba21ba5fa1bb0587 => generated 297 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:27:13.489557 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d8ae26-f040-4098-b44d-42b2c56802df None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:13.549307 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 111/219] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:27:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:27:13.570963 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:13.614444 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d8ae26-f040-4098-b44d-42b2c56802df None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:13.656912 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea53afb-3067-411d-9a17-eccbb1437c1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:13.657531 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea53afb-3067-411d-9a17-eccbb1437c1d None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:13.657807 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea53afb-3067-411d-9a17-eccbb1437c1d None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:13.658080 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea53afb-3067-411d-9a17-eccbb1437c1d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:13.733147 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea53afb-3067-411d-9a17-eccbb1437c1d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:13.734628 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea53afb-3067-411d-9a17-eccbb1437c1d None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:13.805997 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:13.830728 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca1ef51-a426-4f68-858f-0e56cb69c470 None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:13.839646 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 109/220] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Aug 24 17:27:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 17:27:13.846760 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea53afb-3067-411d-9a17-eccbb1437c1d None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:13.909551 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:13.960195 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca1ef51-a426-4f68-858f-0e56cb69c470 None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:13.992587 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea53afb-3067-411d-9a17-eccbb1437c1d None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:13.995816 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9c7775-cd8c-4e3e-98cf-b06dd5b291ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:13.996545 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9c7775-cd8c-4e3e-98cf-b06dd5b291ee None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:13.996936 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9c7775-cd8c-4e3e-98cf-b06dd5b291ee None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:13.997273 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9c7775-cd8c-4e3e-98cf-b06dd5b291ee None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:14.007809 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9c7775-cd8c-4e3e-98cf-b06dd5b291ee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:27:14.013242 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application [None req-9f9c7775-cd8c-4e3e-98cf-b06dd5b291ee None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:27:14.013242 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:27:14.013242 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:27:14.013242 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:27:14.013242 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:27:14.013242 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:27:14.013242 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:27:14.013242 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:27:14.013242 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:27:14.013242 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:27:14.013242 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:27:14.013242 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:27:14.013242 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:27:14.013242 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:27:14.013242 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:27:14.013242 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:27:14.014120 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:27:14.014120 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:27:14.014120 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:27:14.014120 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:27:14.014120 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:27:14.014120 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:27:14.014120 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:27:14.014120 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:27:14.014120 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:27:14.014120 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:27:14.014120 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:27:14.014120 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application with Lock( Aug 24 17:27:14.014120 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:27:14.014120 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:27:14.014120 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:27:14.014120 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:27:14.014910 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:27:14.014910 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:27:14.014910 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:27:14.014910 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:27:14.014910 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:27:14.014910 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:27:14.014910 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:27:14.014910 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:27:14.014910 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:27:14.014910 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:27:14.014910 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:27:14.014910 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Aug 24 17:27:14.019350 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 112/221] 10.222.0.45 () {56 vars in 1096 bytes} [Wed Aug 24 17:27:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 17:27:14.020683 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b133e5b-48ee-4d71-9574-c986c6a44e76 None None] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:14.020953 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b133e5b-48ee-4d71-9574-c986c6a44e76 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:14.021228 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b133e5b-48ee-4d71-9574-c986c6a44e76 None None] PATH_INFO: `/` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:14.023235 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 113/222] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:27:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:27:14.023956 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b133e5b-48ee-4d71-9574-c986c6a44e76 None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:14.035830 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca0321e-cb06-4df4-bd1a-d88e43169181 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:14.036879 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca0321e-cb06-4df4-bd1a-d88e43169181 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:14.037163 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca0321e-cb06-4df4-bd1a-d88e43169181 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:14.037440 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca0321e-cb06-4df4-bd1a-d88e43169181 None admin] PATH_INFO: `/v3/projects/3a29643b449b4fc1aa71802d8db7c0d2` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:14.039195 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca0321e-cb06-4df4-bd1a-d88e43169181 None admin] RBAC: Authorizing `identity:get_project(project_id=3a29643b449b4fc1aa71802d8db7c0d2)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:14.040838 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca0321e-cb06-4df4-bd1a-d88e43169181 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:14.042847 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 110/223] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Aug 24 17:27:13 2022] GET /identity/v3/projects/3a29643b449b4fc1aa71802d8db7c0d2 => generated 302 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:27:14.044000 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188e6f31-a13f-4dab-8c31-47152f1dba7a None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:14.044252 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188e6f31-a13f-4dab-8c31-47152f1dba7a None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:14.044537 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188e6f31-a13f-4dab-8c31-47152f1dba7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:14.065883 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-188e6f31-a13f-4dab-8c31-47152f1dba7a None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:27:14.067254 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-188e6f31-a13f-4dab-8c31-47152f1dba7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:27:14.103929 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:14.145669 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b133e5b-48ee-4d71-9574-c986c6a44e76 None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:14.160001 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:14.162397 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 111/224] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:27:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:27:14.169496 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188e6f31-a13f-4dab-8c31-47152f1dba7a None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:14.193838 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187fc940-70fc-4fa5-8114-bc86721784c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:14.194448 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187fc940-70fc-4fa5-8114-bc86721784c0 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:14.194723 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187fc940-70fc-4fa5-8114-bc86721784c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:14.194994 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187fc940-70fc-4fa5-8114-bc86721784c0 None admin] PATH_INFO: `/v3/roles` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:14.195665 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187fc940-70fc-4fa5-8114-bc86721784c0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:27:14.195965 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187fc940-70fc-4fa5-8114-bc86721784c0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:14.197378 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187fc940-70fc-4fa5-8114-bc86721784c0 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:14.220769 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 114/225] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Aug 24 17:27:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:27:14.222001 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187fc940-70fc-4fa5-8114-bc86721784c0 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:14.366465 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187fc940-70fc-4fa5-8114-bc86721784c0 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:14.382662 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188e6f31-a13f-4dab-8c31-47152f1dba7a None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:14.415417 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30fce5f-c8fd-4fd8-8838-e00e6d29c5d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:14.416060 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30fce5f-c8fd-4fd8-8838-e00e6d29c5d5 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:14.416421 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30fce5f-c8fd-4fd8-8838-e00e6d29c5d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:14.416722 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30fce5f-c8fd-4fd8-8838-e00e6d29c5d5 None admin] PATH_INFO: `/v3/users/2978a18a2c1b4832aeb028089640fd0d` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:14.420562 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30fce5f-c8fd-4fd8-8838-e00e6d29c5d5 None admin] RBAC: Authorizing `identity:get_user(user_id=2978a18a2c1b4832aeb028089640fd0d)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:14.422020 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30fce5f-c8fd-4fd8-8838-e00e6d29c5d5 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:14.424864 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 115/226] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Aug 24 17:27:14 2022] GET /identity/v3/users/2978a18a2c1b4832aeb028089640fd0d => generated 283 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:27:14.431620 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30fce5f-c8fd-4fd8-8838-e00e6d29c5d5 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:14.448867 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e002b7c-979b-4b7c-9db7-c44fed1cde7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:14.449487 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e002b7c-979b-4b7c-9db7-c44fed1cde7f None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:14.449818 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e002b7c-979b-4b7c-9db7-c44fed1cde7f None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:14.450094 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e002b7c-979b-4b7c-9db7-c44fed1cde7f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:14.450826 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e002b7c-979b-4b7c-9db7-c44fed1cde7f None admin] RBAC: Adding query filter params (role.id=02476a5b82eb4b188c18a5d706a3ef68, scope.project.id=3a29643b449b4fc1aa71802d8db7c0d2, user.id=d5e7e3bd287e431eba21ba5fa1bb0587) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:27:14.451134 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e002b7c-979b-4b7c-9db7-c44fed1cde7f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:14.452621 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e002b7c-979b-4b7c-9db7-c44fed1cde7f None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:14.463153 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 112/227] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Aug 24 17:27:14 2022] GET /identity/v3/role_assignments?user.id=d5e7e3bd287e431eba21ba5fa1bb0587&scope.project.id=3a29643b449b4fc1aa71802d8db7c0d2&role.id=02476a5b82eb4b188c18a5d706a3ef68 => generated 259 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:27:14.463854 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e002b7c-979b-4b7c-9db7-c44fed1cde7f None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:14.498333 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:14.501494 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:14.585460 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30fce5f-c8fd-4fd8-8838-e00e6d29c5d5 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:14.630654 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ec2ed2-b5ed-4db3-a893-a347c1709508 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:14.630654 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ec2ed2-b5ed-4db3-a893-a347c1709508 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:14.630654 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ec2ed2-b5ed-4db3-a893-a347c1709508 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:14.630654 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ec2ed2-b5ed-4db3-a893-a347c1709508 None admin] PATH_INFO: `/v3/projects/3a29643b449b4fc1aa71802d8db7c0d2` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:14.632477 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ec2ed2-b5ed-4db3-a893-a347c1709508 None admin] RBAC: Authorizing `identity:get_project(project_id=3a29643b449b4fc1aa71802d8db7c0d2)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:14.635537 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ec2ed2-b5ed-4db3-a893-a347c1709508 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:14.637596 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 116/228] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Aug 24 17:27:14 2022] GET /identity/v3/projects/3a29643b449b4fc1aa71802d8db7c0d2 => generated 302 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:27:14.644644 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ec2ed2-b5ed-4db3-a893-a347c1709508 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:14.691805 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e002b7c-979b-4b7c-9db7-c44fed1cde7f None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:14.748692 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ec2ed2-b5ed-4db3-a893-a347c1709508 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:14.760954 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:14.788426 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c479841b-32b1-407a-8530-54614f45db14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:14.789128 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c479841b-32b1-407a-8530-54614f45db14 None admin] REQUEST_METHOD: `PUT` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:14.789410 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c479841b-32b1-407a-8530-54614f45db14 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:14.789685 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c479841b-32b1-407a-8530-54614f45db14 None admin] PATH_INFO: `/v3/projects/3a29643b449b4fc1aa71802d8db7c0d2/users/2978a18a2c1b4832aeb028089640fd0d/roles/e411ad9eb96e4e8d85b59b5b700ada7b` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:14.791681 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c479841b-32b1-407a-8530-54614f45db14 None admin] RBAC: Authorizing `identity:create_grant(project_id=3a29643b449b4fc1aa71802d8db7c0d2, user_id=2978a18a2c1b4832aeb028089640fd0d, role_id=e411ad9eb96e4e8d85b59b5b700ada7b)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:14.793283 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c479841b-32b1-407a-8530-54614f45db14 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:14.807893 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 117/229] 10.222.0.45 () {58 vars in 1439 bytes} [Wed Aug 24 17:27:14 2022] PUT /identity/v3/projects/3a29643b449b4fc1aa71802d8db7c0d2/users/2978a18a2c1b4832aeb028089640fd0d/roles/e411ad9eb96e4e8d85b59b5b700ada7b => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:27:14.812636 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acfcdb1-2033-448d-9e9e-4a12b7b51792 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:14.812636 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acfcdb1-2033-448d-9e9e-4a12b7b51792 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:14.812636 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acfcdb1-2033-448d-9e9e-4a12b7b51792 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:14.812636 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acfcdb1-2033-448d-9e9e-4a12b7b51792 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:14.836245 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acfcdb1-2033-448d-9e9e-4a12b7b51792 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:27:14.842355 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application [None req-6acfcdb1-2033-448d-9e9e-4a12b7b51792 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 17:27:14.842355 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:27:14.842355 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:27:14.842355 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:27:14.842355 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:27:14.842355 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:27:14.842355 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:27:14.842355 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:27:14.842355 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:27:14.842355 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:27:14.842355 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:27:14.842355 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:27:14.842355 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:27:14.842355 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:27:14.842355 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:27:14.842355 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:27:14.843489 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:27:14.843489 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:27:14.843489 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:27:14.843489 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:27:14.843489 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:27:14.843489 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:27:14.843489 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:27:14.843489 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:27:14.843489 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:27:14.843489 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:27:14.843489 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:27:14.843489 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application with Lock( Aug 24 17:27:14.843489 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:27:14.843489 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:27:14.843489 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:27:14.843489 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:27:14.844301 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:27:14.844301 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:27:14.844301 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:27:14.844301 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:27:14.844301 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:27:14.844301 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:27:14.844301 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:27:14.844301 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:27:14.844301 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:27:14.844301 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:27:14.844301 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 17:27:14.844301 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Aug 24 17:27:14.844301 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 113/230] 10.222.0.45 () {56 vars in 1114 bytes} [Wed Aug 24 17:27:14 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 380 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:27:14.850404 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c479841b-32b1-407a-8530-54614f45db14 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:14.985989 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c479841b-32b1-407a-8530-54614f45db14 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:15.027915 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049bc9e6-92cf-41a3-ab23-d20541ac3cdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:15.028926 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049bc9e6-92cf-41a3-ab23-d20541ac3cdc None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:15.029426 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049bc9e6-92cf-41a3-ab23-d20541ac3cdc None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:15.029911 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049bc9e6-92cf-41a3-ab23-d20541ac3cdc None admin] PATH_INFO: `/v3/roles` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:15.030828 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049bc9e6-92cf-41a3-ab23-d20541ac3cdc None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:27:15.031334 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049bc9e6-92cf-41a3-ab23-d20541ac3cdc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:15.032942 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049bc9e6-92cf-41a3-ab23-d20541ac3cdc None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:15.042618 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 118/231] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Aug 24 17:27:14 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:27:15.048395 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acfcdb1-2033-448d-9e9e-4a12b7b51792 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:15.174636 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:15.188797 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acfcdb1-2033-448d-9e9e-4a12b7b51792 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:15.231509 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f17921e-162f-4835-a7d8-62859863c88b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:15.232218 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f17921e-162f-4835-a7d8-62859863c88b None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:15.232507 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f17921e-162f-4835-a7d8-62859863c88b None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:15.232806 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f17921e-162f-4835-a7d8-62859863c88b None admin] PATH_INFO: `/v3/users/2978a18a2c1b4832aeb028089640fd0d` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:15.235279 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f17921e-162f-4835-a7d8-62859863c88b None admin] RBAC: Authorizing `identity:get_user(user_id=2978a18a2c1b4832aeb028089640fd0d)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:15.236835 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f17921e-162f-4835-a7d8-62859863c88b None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:15.238397 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 114/232] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Aug 24 17:27:15 2022] GET /identity/v3/users/2978a18a2c1b4832aeb028089640fd0d => generated 283 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:27:15.244023 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049bc9e6-92cf-41a3-ab23-d20541ac3cdc None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:15.302520 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:15.399881 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049bc9e6-92cf-41a3-ab23-d20541ac3cdc None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:15.436302 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd336570-77bb-4919-bf02-f1d482c784cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:15.436976 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd336570-77bb-4919-bf02-f1d482c784cd None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:15.437257 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd336570-77bb-4919-bf02-f1d482c784cd None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:15.437531 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd336570-77bb-4919-bf02-f1d482c784cd None admin] PATH_INFO: `/v3/projects/3a29643b449b4fc1aa71802d8db7c0d2` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:15.438675 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd336570-77bb-4919-bf02-f1d482c784cd None admin] RBAC: Authorizing `identity:get_project(project_id=3a29643b449b4fc1aa71802d8db7c0d2)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:15.440126 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd336570-77bb-4919-bf02-f1d482c784cd None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:15.442107 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 119/233] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Aug 24 17:27:15 2022] GET /identity/v3/projects/3a29643b449b4fc1aa71802d8db7c0d2 => generated 302 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:27:15.449597 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f17921e-162f-4835-a7d8-62859863c88b None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:15.565902 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f17921e-162f-4835-a7d8-62859863c88b None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:15.617275 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13f05d7-fc0b-41eb-95d3-1442b76fce68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:15.617951 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13f05d7-fc0b-41eb-95d3-1442b76fce68 None admin] REQUEST_METHOD: `PUT` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:15.618327 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13f05d7-fc0b-41eb-95d3-1442b76fce68 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:15.618613 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13f05d7-fc0b-41eb-95d3-1442b76fce68 None admin] PATH_INFO: `/v3/projects/3a29643b449b4fc1aa71802d8db7c0d2/users/2978a18a2c1b4832aeb028089640fd0d/roles/148f135e4a134697a032be04bc9faa4c` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:15.621648 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13f05d7-fc0b-41eb-95d3-1442b76fce68 None admin] RBAC: Authorizing `identity:create_grant(project_id=3a29643b449b4fc1aa71802d8db7c0d2, user_id=2978a18a2c1b4832aeb028089640fd0d, role_id=148f135e4a134697a032be04bc9faa4c)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:15.623177 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13f05d7-fc0b-41eb-95d3-1442b76fce68 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:15.640655 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 115/234] 10.222.0.45 () {58 vars in 1439 bytes} [Wed Aug 24 17:27:15 2022] PUT /identity/v3/projects/3a29643b449b4fc1aa71802d8db7c0d2/users/2978a18a2c1b4832aeb028089640fd0d/roles/148f135e4a134697a032be04bc9faa4c => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:27:15.833037 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d3d6d5-5634-42ee-a1c9-84bc366e7295 None None] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:15.833550 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d3d6d5-5634-42ee-a1c9-84bc366e7295 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:15.833766 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d3d6d5-5634-42ee-a1c9-84bc366e7295 None None] PATH_INFO: `/` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:15.835036 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 120/235] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:27:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:27:15.845243 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ae8f38-4351-40e0-b8c2-9188b0b4429f None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:15.845739 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ae8f38-4351-40e0-b8c2-9188b0b4429f None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:15.845840 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ae8f38-4351-40e0-b8c2-9188b0b4429f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:15.869554 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-06ae8f38-4351-40e0-b8c2-9188b0b4429f None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:27:15.870736 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-06ae8f38-4351-40e0-b8c2-9188b0b4429f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:27:15.882669 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:15.925865 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5309c3e-3521-4901-8f38-7cd8f9116880 None None] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:15.926143 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5309c3e-3521-4901-8f38-7cd8f9116880 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:15.926425 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5309c3e-3521-4901-8f38-7cd8f9116880 None None] PATH_INFO: `/` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:15.928246 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 121/236] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:27:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 17:27:15.934048 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62cec39-0ee2-4173-8720-2ae8645e0154 None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:15.934745 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62cec39-0ee2-4173-8720-2ae8645e0154 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:15.935034 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62cec39-0ee2-4173-8720-2ae8645e0154 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:15.955595 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 116/237] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:27:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:27:15.960182 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-e62cec39-0ee2-4173-8720-2ae8645e0154 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:27:15.961470 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-e62cec39-0ee2-4173-8720-2ae8645e0154 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:27:16.034000 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 122/238] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:27:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:27:16.276830 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ae8f38-4351-40e0-b8c2-9188b0b4429f None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:16.382793 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62cec39-0ee2-4173-8720-2ae8645e0154 None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:16.411475 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ae8f38-4351-40e0-b8c2-9188b0b4429f None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:16.423456 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:16.454832 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5d725d-fd5a-4e83-a984-a139c3be7f38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:16.455706 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5d725d-fd5a-4e83-a984-a139c3be7f38 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:16.455983 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5d725d-fd5a-4e83-a984-a139c3be7f38 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:16.456251 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5d725d-fd5a-4e83-a984-a139c3be7f38 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:16.476473 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5d725d-fd5a-4e83-a984-a139c3be7f38 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:27:16.482163 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application [None req-aa5d725d-fd5a-4e83-a984-a139c3be7f38 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 17:27:16.482163 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:27:16.482163 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:27:16.482163 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:27:16.482163 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:27:16.482163 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:27:16.482163 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:27:16.482163 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:27:16.482163 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:27:16.482163 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:27:16.482163 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:27:16.482163 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:27:16.482163 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:27:16.482163 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:27:16.482163 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:27:16.482163 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:27:16.483037 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:27:16.483037 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:27:16.483037 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:27:16.483037 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:27:16.483037 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:27:16.483037 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:27:16.483037 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:27:16.483037 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:27:16.483037 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:27:16.483037 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:27:16.483037 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:27:16.483037 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application with Lock( Aug 24 17:27:16.483037 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:27:16.483037 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:27:16.483037 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:27:16.483037 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:27:16.483812 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:27:16.483812 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:27:16.483812 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:27:16.483812 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:27:16.483812 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:27:16.483812 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:27:16.483812 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:27:16.483812 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:27:16.483812 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:27:16.483812 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:27:16.483812 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 17:27:16.483812 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Aug 24 17:27:16.488468 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 117/239] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Aug 24 17:27:16 2022] GET /identity/v3/roles/reader => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:27:16.494036 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5d725d-fd5a-4e83-a984-a139c3be7f38 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:16.510900 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:16.554470 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62cec39-0ee2-4173-8720-2ae8645e0154 None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:16.595588 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa96b3a3-1d12-4188-9ef5-6d50a5cf2599 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:16.596267 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa96b3a3-1d12-4188-9ef5-6d50a5cf2599 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:16.596568 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa96b3a3-1d12-4188-9ef5-6d50a5cf2599 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:16.596902 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa96b3a3-1d12-4188-9ef5-6d50a5cf2599 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:16.598004 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa96b3a3-1d12-4188-9ef5-6d50a5cf2599 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:16.600031 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa96b3a3-1d12-4188-9ef5-6d50a5cf2599 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:16.602019 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 123/240] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 17:27:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:27:16.608153 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa96b3a3-1d12-4188-9ef5-6d50a5cf2599 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:16.715702 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:16.719550 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5d725d-fd5a-4e83-a984-a139c3be7f38 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:16.720268 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa96b3a3-1d12-4188-9ef5-6d50a5cf2599 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:16.779458 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd26855-9fc8-4e54-8cbd-1b57e36b42c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:16.781287 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd26855-9fc8-4e54-8cbd-1b57e36b42c2 None admin] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:16.782238 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd26855-9fc8-4e54-8cbd-1b57e36b42c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:16.783151 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd26855-9fc8-4e54-8cbd-1b57e36b42c2 None admin] PATH_INFO: `/v3/users` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:16.784339 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd26855-9fc8-4e54-8cbd-1b57e36b42c2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:16.786488 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd26855-9fc8-4e54-8cbd-1b57e36b42c2 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:16.787315 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e452aa8b-febc-4760-84b7-3147508c2ef7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:16.788139 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e452aa8b-febc-4760-84b7-3147508c2ef7 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:16.790053 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e452aa8b-febc-4760-84b7-3147508c2ef7 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:16.790535 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e452aa8b-febc-4760-84b7-3147508c2ef7 None admin] PATH_INFO: `/v3/roles` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:16.792034 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e452aa8b-febc-4760-84b7-3147508c2ef7 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:27:16.793756 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e452aa8b-febc-4760-84b7-3147508c2ef7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:16.795792 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e452aa8b-febc-4760-84b7-3147508c2ef7 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:16.808405 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 118/241] 10.222.0.45 () {56 vars in 1101 bytes} [Wed Aug 24 17:27:16 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:27:16.817647 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e452aa8b-febc-4760-84b7-3147508c2ef7 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:16.839446 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 124/242] 10.222.0.45 () {60 vars in 1132 bytes} [Wed Aug 24 17:27:16 2022] POST /identity/v3/users => generated 293 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:27:16.905739 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:16.926597 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7e8620-c7c6-477c-b2fa-46eb5549246d None None] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:16.926961 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7e8620-c7c6-477c-b2fa-46eb5549246d None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:16.927207 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7e8620-c7c6-477c-b2fa-46eb5549246d None None] PATH_INFO: `/` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:16.928571 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 125/243] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:27:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 17:27:16.934973 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747fb1e8-3489-4423-8d47-5fcbb5974dc1 None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:16.935284 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747fb1e8-3489-4423-8d47-5fcbb5974dc1 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:16.935520 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747fb1e8-3489-4423-8d47-5fcbb5974dc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:16.958448 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-747fb1e8-3489-4423-8d47-5fcbb5974dc1 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:27:16.960274 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-747fb1e8-3489-4423-8d47-5fcbb5974dc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:27:16.963799 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e452aa8b-febc-4760-84b7-3147508c2ef7 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:17.040355 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee190fb-57dd-4fc7-bb66-84965d545e71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:17.042759 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee190fb-57dd-4fc7-bb66-84965d545e71 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:17.043272 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee190fb-57dd-4fc7-bb66-84965d545e71 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:17.045492 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee190fb-57dd-4fc7-bb66-84965d545e71 None admin] PATH_INFO: `/v3/users/2ee480620a7d40ad923f7d6e823d7b05` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:17.075574 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee190fb-57dd-4fc7-bb66-84965d545e71 None admin] RBAC: Authorizing `identity:get_user(user_id=2ee480620a7d40ad923f7d6e823d7b05)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:17.077394 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee190fb-57dd-4fc7-bb66-84965d545e71 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:17.079644 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 119/244] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Aug 24 17:27:16 2022] GET /identity/v3/users/2ee480620a7d40ad923f7d6e823d7b05 => generated 297 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:27:17.085406 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee190fb-57dd-4fc7-bb66-84965d545e71 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:17.088590 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 126/245] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:27:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 17:27:17.198599 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee190fb-57dd-4fc7-bb66-84965d545e71 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:17.228246 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:17.237827 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6946df92-715f-4fd1-ba4e-7708ab3e13ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:17.238492 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6946df92-715f-4fd1-ba4e-7708ab3e13ea None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:17.238768 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6946df92-715f-4fd1-ba4e-7708ab3e13ea None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:17.239038 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6946df92-715f-4fd1-ba4e-7708ab3e13ea None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:17.302293 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89f1057-2977-4842-82df-b61c5cb5c791 None None] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:17.302598 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89f1057-2977-4842-82df-b61c5cb5c791 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:17.302854 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89f1057-2977-4842-82df-b61c5cb5c791 None None] PATH_INFO: `/` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:17.304206 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 127/246] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:27:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 17:27:17.311891 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76be16da-31dc-49dd-99c7-a94d0e26f022 None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:17.312176 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76be16da-31dc-49dd-99c7-a94d0e26f022 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:17.312451 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76be16da-31dc-49dd-99c7-a94d0e26f022 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:17.322752 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6946df92-715f-4fd1-ba4e-7708ab3e13ea None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:17.324222 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6946df92-715f-4fd1-ba4e-7708ab3e13ea None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:17.340937 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-76be16da-31dc-49dd-99c7-a94d0e26f022 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:27:17.342986 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-76be16da-31dc-49dd-99c7-a94d0e26f022 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:27:17.441379 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 128/247] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:27:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:27:17.442021 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76be16da-31dc-49dd-99c7-a94d0e26f022 None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:17.446325 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 120/248] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Aug 24 17:27:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 17:27:17.454207 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6946df92-715f-4fd1-ba4e-7708ab3e13ea None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:17.464833 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:17.494352 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:17.585464 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76be16da-31dc-49dd-99c7-a94d0e26f022 None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:17.594869 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6946df92-715f-4fd1-ba4e-7708ab3e13ea None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:17.632788 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed896845-04e2-4472-bb53-b426b8b0d0a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:17.634474 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a863518-0a08-4d9d-8851-a68732da8447 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:17.634852 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed896845-04e2-4472-bb53-b426b8b0d0a4 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:17.635752 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed896845-04e2-4472-bb53-b426b8b0d0a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:17.636127 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed896845-04e2-4472-bb53-b426b8b0d0a4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:17.636235 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a863518-0a08-4d9d-8851-a68732da8447 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:17.637357 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a863518-0a08-4d9d-8851-a68732da8447 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:17.637809 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a863518-0a08-4d9d-8851-a68732da8447 None admin] PATH_INFO: `/v3/projects/3a29643b449b4fc1aa71802d8db7c0d2` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:17.639233 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a863518-0a08-4d9d-8851-a68732da8447 None admin] RBAC: Authorizing `identity:get_project(project_id=3a29643b449b4fc1aa71802d8db7c0d2)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:17.640998 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a863518-0a08-4d9d-8851-a68732da8447 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:17.643188 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 121/249] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Aug 24 17:27:17 2022] GET /identity/v3/projects/3a29643b449b4fc1aa71802d8db7c0d2 => generated 302 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:27:17.649690 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a863518-0a08-4d9d-8851-a68732da8447 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:17.655674 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed896845-04e2-4472-bb53-b426b8b0d0a4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:27:17.661158 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application [None req-ed896845-04e2-4472-bb53-b426b8b0d0a4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 17:27:17.661158 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:27:17.661158 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:27:17.661158 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:27:17.661158 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:27:17.661158 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:27:17.661158 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:27:17.661158 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:27:17.661158 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:27:17.661158 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:27:17.661158 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:27:17.661158 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:27:17.661158 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:27:17.661158 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:27:17.661158 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:27:17.661158 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:27:17.662125 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:27:17.662125 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:27:17.662125 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:27:17.662125 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:27:17.662125 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:27:17.662125 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:27:17.662125 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:27:17.662125 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:27:17.662125 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:27:17.662125 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:27:17.662125 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:27:17.662125 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application with Lock( Aug 24 17:27:17.662125 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:27:17.662125 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:27:17.662125 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:27:17.662125 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:27:17.662940 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:27:17.662940 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:27:17.662940 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:27:17.662940 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:27:17.662940 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:27:17.662940 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:27:17.662940 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:27:17.662940 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:27:17.662940 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:27:17.662940 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:27:17.662940 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 17:27:17.662940 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Aug 24 17:27:17.662940 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 129/250] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Aug 24 17:27:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:27:17.670259 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed896845-04e2-4472-bb53-b426b8b0d0a4 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:17.767338 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:17.786915 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:17.798253 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a863518-0a08-4d9d-8851-a68732da8447 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:17.808143 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed896845-04e2-4472-bb53-b426b8b0d0a4 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:17.839422 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e09b87a-fddd-464a-bb28-1d0b060f684b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:17.840133 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e09b87a-fddd-464a-bb28-1d0b060f684b None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:17.840455 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e09b87a-fddd-464a-bb28-1d0b060f684b None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:17.840769 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e09b87a-fddd-464a-bb28-1d0b060f684b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:17.841630 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e09b87a-fddd-464a-bb28-1d0b060f684b None admin] RBAC: Adding query filter params (role.id=645db6d227604357b1949889630755ff, scope.project.id=3a29643b449b4fc1aa71802d8db7c0d2, user.id=2ee480620a7d40ad923f7d6e823d7b05) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:27:17.841913 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e09b87a-fddd-464a-bb28-1d0b060f684b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:17.843423 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e09b87a-fddd-464a-bb28-1d0b060f684b None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:17.849513 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 122/251] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Aug 24 17:27:17 2022] GET /identity/v3/role_assignments?user.id=2ee480620a7d40ad923f7d6e823d7b05&scope.project.id=3a29643b449b4fc1aa71802d8db7c0d2&role.id=645db6d227604357b1949889630755ff => generated 259 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:27:17.853225 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e09b87a-fddd-464a-bb28-1d0b060f684b None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:17.867456 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0283a8-d948-4b07-8bab-4393aa7903eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:17.869340 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0283a8-d948-4b07-8bab-4393aa7903eb None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:17.869820 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0283a8-d948-4b07-8bab-4393aa7903eb None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:17.870586 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0283a8-d948-4b07-8bab-4393aa7903eb None admin] PATH_INFO: `/v3/roles` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:17.871806 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0283a8-d948-4b07-8bab-4393aa7903eb None admin] RBAC: Adding query filter params (name=member) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:27:17.872261 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0283a8-d948-4b07-8bab-4393aa7903eb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:17.874479 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0283a8-d948-4b07-8bab-4393aa7903eb None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:17.898775 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 130/252] 10.222.0.45 () {56 vars in 1101 bytes} [Wed Aug 24 17:27:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:27:17.905684 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0283a8-d948-4b07-8bab-4393aa7903eb None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:18.083409 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0283a8-d948-4b07-8bab-4393aa7903eb None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:18.102647 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e09b87a-fddd-464a-bb28-1d0b060f684b None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:18.129352 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768b7fee-615a-45ea-bc92-fa37be63989c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:18.130460 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768b7fee-615a-45ea-bc92-fa37be63989c None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:18.130762 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768b7fee-615a-45ea-bc92-fa37be63989c None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:18.131057 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768b7fee-615a-45ea-bc92-fa37be63989c None admin] PATH_INFO: `/v3/users/d5e7e3bd287e431eba21ba5fa1bb0587` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:18.137485 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768b7fee-615a-45ea-bc92-fa37be63989c None admin] RBAC: Authorizing `identity:get_user(user_id=d5e7e3bd287e431eba21ba5fa1bb0587)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:18.138963 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768b7fee-615a-45ea-bc92-fa37be63989c None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:18.144577 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 131/253] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Aug 24 17:27:17 2022] GET /identity/v3/users/d5e7e3bd287e431eba21ba5fa1bb0587 => generated 297 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:27:18.144577 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965ad3c7-80f3-4ed9-9ffc-0ca31ed0b506 None None] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:18.144577 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965ad3c7-80f3-4ed9-9ffc-0ca31ed0b506 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:18.144577 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965ad3c7-80f3-4ed9-9ffc-0ca31ed0b506 None None] PATH_INFO: `/` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:18.144577 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 132/254] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:27:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:27:18.149285 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965ad3c7-80f3-4ed9-9ffc-0ca31ed0b506 None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:18.161656 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096c53d5-911f-49aa-8cd1-564f95577f0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:18.162572 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096c53d5-911f-49aa-8cd1-564f95577f0f None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:18.163688 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096c53d5-911f-49aa-8cd1-564f95577f0f None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:18.163951 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:18.164196 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096c53d5-911f-49aa-8cd1-564f95577f0f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:18.176414 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096c53d5-911f-49aa-8cd1-564f95577f0f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:27:18.189041 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application [None req-096c53d5-911f-49aa-8cd1-564f95577f0f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:27:18.189041 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:27:18.189041 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:27:18.189041 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:27:18.189041 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:27:18.189041 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:27:18.189041 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:27:18.189041 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:27:18.189041 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:27:18.189041 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:27:18.189041 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:27:18.189041 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:27:18.189041 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:27:18.189041 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:27:18.189041 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:27:18.189041 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:27:18.189992 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:27:18.189992 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:27:18.189992 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:27:18.189992 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:27:18.189992 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:27:18.189992 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:27:18.189992 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:27:18.189992 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:27:18.189992 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:27:18.189992 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:27:18.189992 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:27:18.189992 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application with Lock( Aug 24 17:27:18.189992 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:27:18.189992 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:27:18.189992 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:27:18.189992 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:27:18.190764 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:27:18.190764 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:27:18.190764 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:27:18.190764 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:27:18.190764 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:27:18.190764 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:27:18.190764 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:27:18.190764 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:27:18.190764 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:27:18.190764 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:27:18.190764 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:27:18.190764 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Aug 24 17:27:18.190764 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 123/255] 10.222.0.45 () {56 vars in 1096 bytes} [Wed Aug 24 17:27:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 340 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:27:18.191414 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b21a6a2-6c1a-44d2-9a2e-d10305a5e91d None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:18.192072 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b21a6a2-6c1a-44d2-9a2e-d10305a5e91d None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:18.192072 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b21a6a2-6c1a-44d2-9a2e-d10305a5e91d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:18.229431 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-9b21a6a2-6c1a-44d2-9a2e-d10305a5e91d None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:27:18.231586 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-9b21a6a2-6c1a-44d2-9a2e-d10305a5e91d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:27:18.265424 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:18.328181 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965ad3c7-80f3-4ed9-9ffc-0ca31ed0b506 None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:18.360593 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 124/256] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:27:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:27:18.362024 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b21a6a2-6c1a-44d2-9a2e-d10305a5e91d None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:18.444575 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29be996-5383-4b8a-a770-b5cf78d92030 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:18.444575 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29be996-5383-4b8a-a770-b5cf78d92030 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:18.444575 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29be996-5383-4b8a-a770-b5cf78d92030 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:18.444575 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29be996-5383-4b8a-a770-b5cf78d92030 None admin] PATH_INFO: `/v3/projects/3a29643b449b4fc1aa71802d8db7c0d2` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:18.444967 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29be996-5383-4b8a-a770-b5cf78d92030 None admin] RBAC: Authorizing `identity:get_project(project_id=3a29643b449b4fc1aa71802d8db7c0d2)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:18.444967 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29be996-5383-4b8a-a770-b5cf78d92030 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:18.444967 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 133/257] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Aug 24 17:27:18 2022] GET /identity/v3/projects/3a29643b449b4fc1aa71802d8db7c0d2 => generated 302 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:27:18.449231 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29be996-5383-4b8a-a770-b5cf78d92030 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:18.526634 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b21a6a2-6c1a-44d2-9a2e-d10305a5e91d None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:18.564928 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:18.590247 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175ae79a-2a67-42c3-a09e-2df2616d0659 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:18.590997 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175ae79a-2a67-42c3-a09e-2df2616d0659 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:18.591342 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175ae79a-2a67-42c3-a09e-2df2616d0659 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:18.591685 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175ae79a-2a67-42c3-a09e-2df2616d0659 None admin] PATH_INFO: `/v3/roles` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:18.592431 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175ae79a-2a67-42c3-a09e-2df2616d0659 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:27:18.592824 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175ae79a-2a67-42c3-a09e-2df2616d0659 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:18.594351 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175ae79a-2a67-42c3-a09e-2df2616d0659 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:18.608835 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29be996-5383-4b8a-a770-b5cf78d92030 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:18.617451 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 125/258] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Aug 24 17:27:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:27:18.623368 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175ae79a-2a67-42c3-a09e-2df2616d0659 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:18.646510 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:18.673924 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc654ba-e7a9-474d-93ae-621a76bdbd48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:18.674612 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc654ba-e7a9-474d-93ae-621a76bdbd48 None admin] REQUEST_METHOD: `PUT` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:18.674906 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc654ba-e7a9-474d-93ae-621a76bdbd48 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:18.675159 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc654ba-e7a9-474d-93ae-621a76bdbd48 None admin] PATH_INFO: `/v3/projects/3a29643b449b4fc1aa71802d8db7c0d2/users/d5e7e3bd287e431eba21ba5fa1bb0587/roles/02476a5b82eb4b188c18a5d706a3ef68` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:18.677127 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc654ba-e7a9-474d-93ae-621a76bdbd48 None admin] RBAC: Authorizing `identity:create_grant(project_id=3a29643b449b4fc1aa71802d8db7c0d2, user_id=d5e7e3bd287e431eba21ba5fa1bb0587, role_id=02476a5b82eb4b188c18a5d706a3ef68)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:18.678581 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc654ba-e7a9-474d-93ae-621a76bdbd48 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:18.709580 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 134/259] 10.222.0.45 () {58 vars in 1439 bytes} [Wed Aug 24 17:27:18 2022] PUT /identity/v3/projects/3a29643b449b4fc1aa71802d8db7c0d2/users/d5e7e3bd287e431eba21ba5fa1bb0587/roles/02476a5b82eb4b188c18a5d706a3ef68 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:27:18.826087 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175ae79a-2a67-42c3-a09e-2df2616d0659 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:18.838856 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc654ba-e7a9-474d-93ae-621a76bdbd48 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:18.886094 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ad6dfb-72fe-48e0-81a7-7d1a8080d1f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:18.886668 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ad6dfb-72fe-48e0-81a7-7d1a8080d1f1 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:18.886945 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ad6dfb-72fe-48e0-81a7-7d1a8080d1f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:18.887216 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ad6dfb-72fe-48e0-81a7-7d1a8080d1f1 None admin] PATH_INFO: `/v3/users/2978a18a2c1b4832aeb028089640fd0d` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:18.889832 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ad6dfb-72fe-48e0-81a7-7d1a8080d1f1 None admin] RBAC: Authorizing `identity:get_user(user_id=2978a18a2c1b4832aeb028089640fd0d)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:18.891316 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ad6dfb-72fe-48e0-81a7-7d1a8080d1f1 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:18.902949 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 126/260] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Aug 24 17:27:18 2022] GET /identity/v3/users/2978a18a2c1b4832aeb028089640fd0d => generated 283 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:27:18.913665 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ad6dfb-72fe-48e0-81a7-7d1a8080d1f1 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:19.005054 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc654ba-e7a9-474d-93ae-621a76bdbd48 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:19.038456 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:19.047852 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c41ef7-efed-42f9-9cf4-4b16b780612d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:19.049032 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c41ef7-efed-42f9-9cf4-4b16b780612d None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:19.049535 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c41ef7-efed-42f9-9cf4-4b16b780612d None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:19.050023 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c41ef7-efed-42f9-9cf4-4b16b780612d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:19.058306 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:19.071513 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c41ef7-efed-42f9-9cf4-4b16b780612d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:27:19.082295 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application [None req-42c41ef7-efed-42f9-9cf4-4b16b780612d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 17:27:19.082295 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:27:19.082295 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:27:19.082295 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:27:19.082295 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:27:19.082295 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:27:19.082295 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:27:19.082295 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:27:19.082295 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:27:19.082295 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:27:19.082295 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:27:19.082295 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:27:19.082295 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:27:19.082295 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:27:19.082295 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:27:19.082295 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:27:19.083204 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:27:19.083204 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:27:19.083204 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:27:19.083204 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:27:19.083204 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:27:19.083204 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:27:19.083204 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:27:19.083204 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:27:19.083204 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:27:19.083204 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:27:19.083204 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:27:19.083204 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application with Lock( Aug 24 17:27:19.083204 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:27:19.083204 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:27:19.083204 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:27:19.083204 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:27:19.083985 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:27:19.083985 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:27:19.083985 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:27:19.083985 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:27:19.083985 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:27:19.083985 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:27:19.083985 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:27:19.083985 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:27:19.083985 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:27:19.083985 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:27:19.083985 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 17:27:19.083985 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Aug 24 17:27:19.086655 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 135/261] 10.222.0.45 () {56 vars in 1114 bytes} [Wed Aug 24 17:27:18 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:27:19.093854 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c41ef7-efed-42f9-9cf4-4b16b780612d None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:19.107956 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ad6dfb-72fe-48e0-81a7-7d1a8080d1f1 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:19.170216 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a991027-6bd5-43eb-b9f8-697f494e1a1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:19.170848 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a991027-6bd5-43eb-b9f8-697f494e1a1e None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:19.171133 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a991027-6bd5-43eb-b9f8-697f494e1a1e None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:19.171405 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a991027-6bd5-43eb-b9f8-697f494e1a1e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:19.250743 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c41ef7-efed-42f9-9cf4-4b16b780612d None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:19.285832 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a991027-6bd5-43eb-b9f8-697f494e1a1e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:19.287373 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a991027-6bd5-43eb-b9f8-697f494e1a1e None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:19.291963 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4caf92d1-067c-45c0-a832-e3d5d022fbfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:19.292618 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caf92d1-067c-45c0-a832-e3d5d022fbfd None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:19.292900 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caf92d1-067c-45c0-a832-e3d5d022fbfd None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:19.293169 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caf92d1-067c-45c0-a832-e3d5d022fbfd None admin] PATH_INFO: `/v3/roles` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:19.293851 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4caf92d1-067c-45c0-a832-e3d5d022fbfd None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:27:19.294151 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4caf92d1-067c-45c0-a832-e3d5d022fbfd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:19.298744 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4caf92d1-067c-45c0-a832-e3d5d022fbfd None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:19.312639 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 136/262] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Aug 24 17:27:19 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:27:19.318732 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4caf92d1-067c-45c0-a832-e3d5d022fbfd None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:19.419612 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:19.449686 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:19.495528 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 127/263] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Aug 24 17:27:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 582 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 17:27:19.501634 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a991027-6bd5-43eb-b9f8-697f494e1a1e None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:19.507864 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4caf92d1-067c-45c0-a832-e3d5d022fbfd None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:19.571942 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03db5992-ef66-40d9-978b-8753b6dbf6c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:19.573014 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03db5992-ef66-40d9-978b-8753b6dbf6c3 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:19.573446 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03db5992-ef66-40d9-978b-8753b6dbf6c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:19.573870 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03db5992-ef66-40d9-978b-8753b6dbf6c3 None admin] PATH_INFO: `/v3/users/2978a18a2c1b4832aeb028089640fd0d` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:19.576410 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03db5992-ef66-40d9-978b-8753b6dbf6c3 None admin] RBAC: Authorizing `identity:get_user(user_id=2978a18a2c1b4832aeb028089640fd0d)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:19.578367 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03db5992-ef66-40d9-978b-8753b6dbf6c3 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:19.580577 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 137/264] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Aug 24 17:27:19 2022] GET /identity/v3/users/2978a18a2c1b4832aeb028089640fd0d => generated 283 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:27:19.585694 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03db5992-ef66-40d9-978b-8753b6dbf6c3 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:19.615856 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a991027-6bd5-43eb-b9f8-697f494e1a1e None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:19.655570 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33773d5-3213-4760-ba62-9997a7618156 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:19.656207 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33773d5-3213-4760-ba62-9997a7618156 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:19.656485 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33773d5-3213-4760-ba62-9997a7618156 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:19.656785 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33773d5-3213-4760-ba62-9997a7618156 None admin] PATH_INFO: `/v3/projects/3a29643b449b4fc1aa71802d8db7c0d2` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:19.658795 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33773d5-3213-4760-ba62-9997a7618156 None admin] RBAC: Authorizing `identity:get_project(project_id=3a29643b449b4fc1aa71802d8db7c0d2)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:19.660362 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33773d5-3213-4760-ba62-9997a7618156 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:19.662343 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 128/265] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Aug 24 17:27:19 2022] GET /identity/v3/projects/3a29643b449b4fc1aa71802d8db7c0d2 => generated 302 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:27:19.669999 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33773d5-3213-4760-ba62-9997a7618156 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:19.745826 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:19.810524 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33773d5-3213-4760-ba62-9997a7618156 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:19.819489 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03db5992-ef66-40d9-978b-8753b6dbf6c3 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:19.851309 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a304e0-9a96-4957-af36-f9bb95c36580 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:19.852068 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a304e0-9a96-4957-af36-f9bb95c36580 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:19.852446 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a304e0-9a96-4957-af36-f9bb95c36580 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:19.853106 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a304e0-9a96-4957-af36-f9bb95c36580 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:19.854504 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a304e0-9a96-4957-af36-f9bb95c36580 None admin] RBAC: Adding query filter params (role.id=e411ad9eb96e4e8d85b59b5b700ada7b, scope.project.id=3a29643b449b4fc1aa71802d8db7c0d2, user.id=2978a18a2c1b4832aeb028089640fd0d) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:27:19.854910 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a304e0-9a96-4957-af36-f9bb95c36580 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:19.857055 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a304e0-9a96-4957-af36-f9bb95c36580 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:19.860539 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:19.866712 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 129/266] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Aug 24 17:27:19 2022] GET /identity/v3/role_assignments?user.id=2978a18a2c1b4832aeb028089640fd0d&scope.project.id=3a29643b449b4fc1aa71802d8db7c0d2&role.id=e411ad9eb96e4e8d85b59b5b700ada7b => generated 608 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:27:19.894662 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a27c594-4a41-4432-b28c-c887b59c33a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:19.895295 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a27c594-4a41-4432-b28c-c887b59c33a8 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:19.895580 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a27c594-4a41-4432-b28c-c887b59c33a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:19.895857 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a27c594-4a41-4432-b28c-c887b59c33a8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:19.924123 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6caa72-e077-4aa8-ae85-517c36aa66ce None None] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:19.924429 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6caa72-e077-4aa8-ae85-517c36aa66ce None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:19.924708 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6caa72-e077-4aa8-ae85-517c36aa66ce None None] PATH_INFO: `/` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:19.926327 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 130/267] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:27:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 17:27:19.932356 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdfb4f7-0be3-4234-81a0-fd8e2a39a10d None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:19.932675 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdfb4f7-0be3-4234-81a0-fd8e2a39a10d None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:19.932943 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdfb4f7-0be3-4234-81a0-fd8e2a39a10d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:19.968356 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-1bdfb4f7-0be3-4234-81a0-fd8e2a39a10d None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:27:19.971348 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-1bdfb4f7-0be3-4234-81a0-fd8e2a39a10d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:27:20.021838 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a27c594-4a41-4432-b28c-c887b59c33a8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:20.023988 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a27c594-4a41-4432-b28c-c887b59c33a8 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:20.144005 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 131/268] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:27:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:27:20.247683 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 138/269] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Aug 24 17:27:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 662 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 17:27:20.253486 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdfb4f7-0be3-4234-81a0-fd8e2a39a10d None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:20.336409 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:20.416819 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdfb4f7-0be3-4234-81a0-fd8e2a39a10d None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:20.483978 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15899fc-b19f-4733-a465-5fa654823284 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:20.484632 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ac142e-c74b-4d4b-861c-3eee405ce22a None None] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:20.484744 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15899fc-b19f-4733-a465-5fa654823284 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:20.484925 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15899fc-b19f-4733-a465-5fa654823284 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:20.485199 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15899fc-b19f-4733-a465-5fa654823284 None admin] PATH_INFO: `/v3/projects/3a29643b449b4fc1aa71802d8db7c0d2` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:20.485611 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ac142e-c74b-4d4b-861c-3eee405ce22a None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:20.486094 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ac142e-c74b-4d4b-861c-3eee405ce22a None None] PATH_INFO: `/` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:20.487627 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 139/270] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:27:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:27:20.489305 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15899fc-b19f-4733-a465-5fa654823284 None admin] RBAC: Authorizing `identity:get_project(project_id=3a29643b449b4fc1aa71802d8db7c0d2)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:20.490796 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15899fc-b19f-4733-a465-5fa654823284 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:20.492917 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 132/271] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Aug 24 17:27:20 2022] GET /identity/v3/projects/3a29643b449b4fc1aa71802d8db7c0d2 => generated 302 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:27:20.493914 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56517561-f72f-44fb-84fd-ef79e7a0f885 None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:20.494944 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56517561-f72f-44fb-84fd-ef79e7a0f885 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:20.495420 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56517561-f72f-44fb-84fd-ef79e7a0f885 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:20.499430 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15899fc-b19f-4733-a465-5fa654823284 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:20.517746 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-56517561-f72f-44fb-84fd-ef79e7a0f885 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:27:20.519339 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-56517561-f72f-44fb-84fd-ef79e7a0f885 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:27:20.544245 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:20.606590 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 140/272] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:27:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:27:20.699334 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf68154-77be-4a4b-8b66-61a02a4e3151 None None] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:20.699850 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf68154-77be-4a4b-8b66-61a02a4e3151 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:20.700267 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf68154-77be-4a4b-8b66-61a02a4e3151 None None] PATH_INFO: `/` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:20.701768 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 141/273] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:27:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 17:27:20.714232 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ea8646-cd57-4608-ae87-331b169a534f None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:20.714508 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ea8646-cd57-4608-ae87-331b169a534f None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:20.714957 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ea8646-cd57-4608-ae87-331b169a534f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:20.736077 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-90ea8646-cd57-4608-ae87-331b169a534f None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:27:20.737279 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-90ea8646-cd57-4608-ae87-331b169a534f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:27:20.738589 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15899fc-b19f-4733-a465-5fa654823284 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:20.756313 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:20.803890 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ab671f-51bd-4893-96a4-21b4b1aadb4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:20.804896 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ab671f-51bd-4893-96a4-21b4b1aadb4d None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:20.805442 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ab671f-51bd-4893-96a4-21b4b1aadb4d None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:20.805923 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ab671f-51bd-4893-96a4-21b4b1aadb4d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:20.806933 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ab671f-51bd-4893-96a4-21b4b1aadb4d None admin] RBAC: Adding query filter params (role.id=148f135e4a134697a032be04bc9faa4c, scope.project.id=3a29643b449b4fc1aa71802d8db7c0d2, user.id=2978a18a2c1b4832aeb028089640fd0d) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:27:20.807438 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ab671f-51bd-4893-96a4-21b4b1aadb4d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:20.809313 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ab671f-51bd-4893-96a4-21b4b1aadb4d None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:20.816293 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 133/274] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Aug 24 17:27:20 2022] GET /identity/v3/role_assignments?user.id=2978a18a2c1b4832aeb028089640fd0d&scope.project.id=3a29643b449b4fc1aa71802d8db7c0d2&role.id=148f135e4a134697a032be04bc9faa4c => generated 608 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:27:20.817299 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ab671f-51bd-4893-96a4-21b4b1aadb4d None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:20.868672 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 142/275] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:27:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:27:20.925703 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ea8646-cd57-4608-ae87-331b169a534f None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:20.973370 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ab671f-51bd-4893-96a4-21b4b1aadb4d None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:21.028483 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9120e4-54f5-4fae-b61f-048a22bf242e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:21.029191 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9120e4-54f5-4fae-b61f-048a22bf242e None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:21.029474 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9120e4-54f5-4fae-b61f-048a22bf242e None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:21.029851 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9120e4-54f5-4fae-b61f-048a22bf242e None admin] PATH_INFO: `/v3/roles/member` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:21.033687 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:21.044010 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9120e4-54f5-4fae-b61f-048a22bf242e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:27:21.050360 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application [None req-ca9120e4-54f5-4fae-b61f-048a22bf242e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 17:27:21.050360 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:27:21.050360 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:27:21.050360 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:27:21.050360 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:27:21.050360 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:27:21.050360 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:27:21.050360 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:27:21.050360 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:27:21.050360 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:27:21.050360 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:27:21.050360 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:27:21.050360 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:27:21.050360 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:27:21.050360 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:27:21.050360 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:27:21.051213 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:27:21.051213 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:27:21.051213 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:27:21.051213 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:27:21.051213 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:27:21.051213 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:27:21.051213 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:27:21.051213 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:27:21.051213 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:27:21.051213 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:27:21.051213 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:27:21.051213 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application with Lock( Aug 24 17:27:21.051213 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:27:21.051213 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:27:21.051213 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:27:21.051213 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:27:21.052630 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:27:21.052630 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:27:21.052630 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:27:21.052630 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:27:21.052630 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:27:21.052630 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:27:21.052630 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:27:21.052630 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:27:21.052630 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:27:21.052630 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:27:21.052630 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 17:27:21.052630 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Aug 24 17:27:21.052630 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 134/276] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Aug 24 17:27:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:27:21.057965 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9120e4-54f5-4fae-b61f-048a22bf242e None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:21.069001 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ea8646-cd57-4608-ae87-331b169a534f None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:21.099941 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:21.116022 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038c483d-58c5-45ba-88c1-3e3dbc81c019 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:21.116712 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038c483d-58c5-45ba-88c1-3e3dbc81c019 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:21.116993 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038c483d-58c5-45ba-88c1-3e3dbc81c019 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:21.117266 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038c483d-58c5-45ba-88c1-3e3dbc81c019 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:21.120141 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038c483d-58c5-45ba-88c1-3e3dbc81c019 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:21.121662 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038c483d-58c5-45ba-88c1-3e3dbc81c019 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:21.123652 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 143/277] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 17:27:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:27:21.129117 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038c483d-58c5-45ba-88c1-3e3dbc81c019 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:21.177582 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9120e4-54f5-4fae-b61f-048a22bf242e None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:21.213014 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b12be18-302c-4b6a-9c3b-c0bdbc930c86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:21.214505 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b12be18-302c-4b6a-9c3b-c0bdbc930c86 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:21.214997 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b12be18-302c-4b6a-9c3b-c0bdbc930c86 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:21.215455 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b12be18-302c-4b6a-9c3b-c0bdbc930c86 None admin] PATH_INFO: `/v3/roles` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:21.216309 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b12be18-302c-4b6a-9c3b-c0bdbc930c86 None admin] RBAC: Adding query filter params (name=member) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:27:21.216905 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b12be18-302c-4b6a-9c3b-c0bdbc930c86 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:21.218474 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b12be18-302c-4b6a-9c3b-c0bdbc930c86 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:21.228728 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 135/278] 10.222.0.45 () {56 vars in 1101 bytes} [Wed Aug 24 17:27:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:27:21.234281 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b12be18-302c-4b6a-9c3b-c0bdbc930c86 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:21.256583 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038c483d-58c5-45ba-88c1-3e3dbc81c019 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:21.292613 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cabee12-c7c4-4fd0-9af9-9d8f5acc0f66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:21.293464 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cabee12-c7c4-4fd0-9af9-9d8f5acc0f66 None admin] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:21.293740 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cabee12-c7c4-4fd0-9af9-9d8f5acc0f66 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:21.294004 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cabee12-c7c4-4fd0-9af9-9d8f5acc0f66 None admin] PATH_INFO: `/v3/users` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:21.294740 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cabee12-c7c4-4fd0-9af9-9d8f5acc0f66 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:21.296171 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cabee12-c7c4-4fd0-9af9-9d8f5acc0f66 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:21.320140 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 144/279] 10.222.0.45 () {60 vars in 1132 bytes} [Wed Aug 24 17:27:21 2022] POST /identity/v3/users => generated 293 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:27:21.320910 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cabee12-c7c4-4fd0-9af9-9d8f5acc0f66 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:21.332279 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:21.370491 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:21.384286 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b12be18-302c-4b6a-9c3b-c0bdbc930c86 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:21.423568 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef87a15c-f483-412f-9e01-32857a7125f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:21.424312 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef87a15c-f483-412f-9e01-32857a7125f0 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:21.424615 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef87a15c-f483-412f-9e01-32857a7125f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:21.424891 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef87a15c-f483-412f-9e01-32857a7125f0 None admin] PATH_INFO: `/v3/users/a864394f741741a49f3d25e30f38c231` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:21.440356 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef87a15c-f483-412f-9e01-32857a7125f0 None admin] RBAC: Authorizing `identity:get_user(user_id=a864394f741741a49f3d25e30f38c231)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:21.441909 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef87a15c-f483-412f-9e01-32857a7125f0 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:21.444066 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 136/280] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Aug 24 17:27:21 2022] GET /identity/v3/users/a864394f741741a49f3d25e30f38c231 => generated 293 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:27:21.450948 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef87a15c-f483-412f-9e01-32857a7125f0 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:21.457031 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cabee12-c7c4-4fd0-9af9-9d8f5acc0f66 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:21.507568 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6f2a38-1808-4f77-846d-f7931f25d274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:21.508803 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6f2a38-1808-4f77-846d-f7931f25d274 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:21.509241 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6f2a38-1808-4f77-846d-f7931f25d274 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:21.509911 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6f2a38-1808-4f77-846d-f7931f25d274 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:21.527743 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6f2a38-1808-4f77-846d-f7931f25d274 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:27:21.534360 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application [None req-ec6f2a38-1808-4f77-846d-f7931f25d274 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 17:27:21.534360 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:27:21.534360 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:27:21.534360 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:27:21.534360 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:27:21.534360 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:27:21.534360 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:27:21.534360 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:27:21.534360 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:27:21.534360 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:27:21.534360 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:27:21.534360 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:27:21.534360 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:27:21.534360 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:27:21.534360 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:27:21.534360 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:27:21.535256 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:27:21.535256 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:27:21.535256 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:27:21.535256 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:27:21.535256 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:27:21.535256 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:27:21.535256 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:27:21.535256 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:27:21.535256 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:27:21.535256 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:27:21.535256 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:27:21.535256 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application with Lock( Aug 24 17:27:21.535256 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:27:21.535256 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:27:21.535256 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:27:21.535256 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:27:21.536040 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:27:21.536040 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:27:21.536040 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:27:21.536040 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:27:21.536040 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:27:21.536040 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:27:21.536040 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:27:21.536040 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:27:21.536040 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:27:21.536040 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:27:21.536040 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 17:27:21.536040 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Aug 24 17:27:21.538413 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 145/281] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Aug 24 17:27:21 2022] GET /identity/v3/roles/reader => generated 84 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:27:21.544693 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6f2a38-1808-4f77-846d-f7931f25d274 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:21.581355 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef87a15c-f483-412f-9e01-32857a7125f0 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:21.627715 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869e6613-d278-414b-83b4-46ecf7443ed7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:21.628268 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869e6613-d278-414b-83b4-46ecf7443ed7 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:21.628545 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869e6613-d278-414b-83b4-46ecf7443ed7 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:21.628876 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869e6613-d278-414b-83b4-46ecf7443ed7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:21.629612 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869e6613-d278-414b-83b4-46ecf7443ed7 None admin] RBAC: Adding query filter params (role.id=02476a5b82eb4b188c18a5d706a3ef68, user.id=a864394f741741a49f3d25e30f38c231, scope.system=all) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:27:21.629922 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869e6613-d278-414b-83b4-46ecf7443ed7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:21.631392 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869e6613-d278-414b-83b4-46ecf7443ed7 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:21.632160 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:21.638597 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 137/282] 10.222.0.45 () {56 vars in 1308 bytes} [Wed Aug 24 17:27:21 2022] GET /identity/v3/role_assignments?user.id=a864394f741741a49f3d25e30f38c231&scope.system=all&role.id=02476a5b82eb4b188c18a5d706a3ef68 => generated 226 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:27:21.656368 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35be22a8-ef86-4a92-b12a-3b99b2c4e495 None None] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:21.656837 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35be22a8-ef86-4a92-b12a-3b99b2c4e495 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:21.657064 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35be22a8-ef86-4a92-b12a-3b99b2c4e495 None None] PATH_INFO: `/` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:21.658540 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 138/283] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:27:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 17:27:21.664992 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f34467-4504-4cb5-8b13-cea23c6ff8d1 None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:21.665382 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f34467-4504-4cb5-8b13-cea23c6ff8d1 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:21.665663 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f34467-4504-4cb5-8b13-cea23c6ff8d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:21.678042 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6f2a38-1808-4f77-846d-f7931f25d274 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:21.688640 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-37f34467-4504-4cb5-8b13-cea23c6ff8d1 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:27:21.689863 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-37f34467-4504-4cb5-8b13-cea23c6ff8d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:27:21.703459 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:21.723311 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d66226-8cba-4926-8934-c33366ec8a17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:21.724001 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d66226-8cba-4926-8934-c33366ec8a17 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:21.724329 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d66226-8cba-4926-8934-c33366ec8a17 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:21.724652 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d66226-8cba-4926-8934-c33366ec8a17 None admin] PATH_INFO: `/v3/roles` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:21.725397 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d66226-8cba-4926-8934-c33366ec8a17 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:27:21.725704 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d66226-8cba-4926-8934-c33366ec8a17 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:21.727242 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d66226-8cba-4926-8934-c33366ec8a17 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:21.739049 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 146/284] 10.222.0.45 () {56 vars in 1101 bytes} [Wed Aug 24 17:27:21 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:27:21.744403 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d66226-8cba-4926-8934-c33366ec8a17 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:21.786216 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 139/285] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:27:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:27:21.864232 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d66226-8cba-4926-8934-c33366ec8a17 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:21.891193 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:21.899717 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2e09cd-7c28-4acf-902a-5073d6f86b7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:21.900359 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2e09cd-7c28-4acf-902a-5073d6f86b7b None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:21.901231 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2e09cd-7c28-4acf-902a-5073d6f86b7b None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:21.901511 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2e09cd-7c28-4acf-902a-5073d6f86b7b None admin] PATH_INFO: `/v3/users/2ee480620a7d40ad923f7d6e823d7b05` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:21.902710 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2e09cd-7c28-4acf-902a-5073d6f86b7b None admin] RBAC: Authorizing `identity:get_user(user_id=2ee480620a7d40ad923f7d6e823d7b05)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:21.904171 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2e09cd-7c28-4acf-902a-5073d6f86b7b None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:21.906091 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 147/286] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Aug 24 17:27:21 2022] GET /identity/v3/users/2ee480620a7d40ad923f7d6e823d7b05 => generated 297 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:27:21.913872 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f34467-4504-4cb5-8b13-cea23c6ff8d1 None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:22.061125 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f34467-4504-4cb5-8b13-cea23c6ff8d1 None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:22.068149 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2e09cd-7c28-4acf-902a-5073d6f86b7b None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:22.126432 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e353eb4-2ef8-4e87-900c-94079d432586 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:22.127072 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e353eb4-2ef8-4e87-900c-94079d432586 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:22.127346 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e353eb4-2ef8-4e87-900c-94079d432586 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:22.127615 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e353eb4-2ef8-4e87-900c-94079d432586 None admin] PATH_INFO: `/v3/projects/3a29643b449b4fc1aa71802d8db7c0d2` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:22.129911 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e353eb4-2ef8-4e87-900c-94079d432586 None admin] RBAC: Authorizing `identity:get_project(project_id=3a29643b449b4fc1aa71802d8db7c0d2)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:22.131357 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e353eb4-2ef8-4e87-900c-94079d432586 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:22.133016 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 140/287] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Aug 24 17:27:21 2022] GET /identity/v3/projects/3a29643b449b4fc1aa71802d8db7c0d2 => generated 302 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:27:22.139517 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e353eb4-2ef8-4e87-900c-94079d432586 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:22.170058 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:22.222792 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2e09cd-7c28-4acf-902a-5073d6f86b7b None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:22.258513 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaef38c1-7b02-446b-9c46-7248bfe82615 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:22.259158 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaef38c1-7b02-446b-9c46-7248bfe82615 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:22.259440 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaef38c1-7b02-446b-9c46-7248bfe82615 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:22.259713 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaef38c1-7b02-446b-9c46-7248bfe82615 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:22.270107 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaef38c1-7b02-446b-9c46-7248bfe82615 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:27:22.275355 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application [None req-aaef38c1-7b02-446b-9c46-7248bfe82615 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 17:27:22.275355 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:27:22.275355 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:27:22.275355 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:27:22.275355 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:27:22.275355 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:27:22.275355 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:27:22.275355 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:27:22.275355 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:27:22.275355 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:27:22.275355 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:27:22.275355 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:27:22.275355 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:27:22.275355 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:27:22.275355 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:27:22.275355 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:27:22.276252 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:27:22.276252 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:27:22.276252 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:27:22.276252 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:27:22.276252 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:27:22.276252 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:27:22.276252 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:27:22.276252 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:27:22.276252 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:27:22.276252 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:27:22.276252 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:27:22.276252 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application with Lock( Aug 24 17:27:22.276252 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:27:22.276252 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:27:22.276252 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:27:22.276252 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:27:22.277111 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:27:22.277111 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:27:22.277111 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:27:22.277111 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:27:22.277111 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:27:22.277111 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:27:22.277111 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:27:22.277111 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:27:22.277111 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:27:22.277111 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:27:22.277111 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 17:27:22.277111 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Aug 24 17:27:22.277111 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 148/288] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Aug 24 17:27:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 17:27:22.284505 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaef38c1-7b02-446b-9c46-7248bfe82615 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:22.340958 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e353eb4-2ef8-4e87-900c-94079d432586 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:22.345561 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:22.391481 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaef38c1-7b02-446b-9c46-7248bfe82615 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:22.405837 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e62b1eb-3219-4f2a-b750-63d5179ea18d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:22.406560 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e62b1eb-3219-4f2a-b750-63d5179ea18d None admin] REQUEST_METHOD: `PUT` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:22.406882 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e62b1eb-3219-4f2a-b750-63d5179ea18d None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:22.407160 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e62b1eb-3219-4f2a-b750-63d5179ea18d None admin] PATH_INFO: `/v3/projects/3a29643b449b4fc1aa71802d8db7c0d2/users/2ee480620a7d40ad923f7d6e823d7b05/roles/645db6d227604357b1949889630755ff` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:22.411937 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e62b1eb-3219-4f2a-b750-63d5179ea18d None admin] RBAC: Authorizing `identity:create_grant(project_id=3a29643b449b4fc1aa71802d8db7c0d2, user_id=2ee480620a7d40ad923f7d6e823d7b05, role_id=645db6d227604357b1949889630755ff)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:22.413409 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e62b1eb-3219-4f2a-b750-63d5179ea18d None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:22.426634 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724306ff-f826-48d0-941d-7415629435df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:22.427502 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724306ff-f826-48d0-941d-7415629435df None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:22.427824 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724306ff-f826-48d0-941d-7415629435df None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:22.428106 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724306ff-f826-48d0-941d-7415629435df None admin] PATH_INFO: `/v3/roles` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:22.429213 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724306ff-f826-48d0-941d-7415629435df None admin] RBAC: Adding query filter params (name=member) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:27:22.429518 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724306ff-f826-48d0-941d-7415629435df None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:22.431224 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724306ff-f826-48d0-941d-7415629435df None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:22.435507 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 141/289] 10.222.0.45 () {58 vars in 1439 bytes} [Wed Aug 24 17:27:22 2022] PUT /identity/v3/projects/3a29643b449b4fc1aa71802d8db7c0d2/users/2ee480620a7d40ad923f7d6e823d7b05/roles/645db6d227604357b1949889630755ff => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:27:22.440584 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 149/290] 10.222.0.45 () {56 vars in 1101 bytes} [Wed Aug 24 17:27:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:27:22.449981 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e62b1eb-3219-4f2a-b750-63d5179ea18d None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:22.551509 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:22.582879 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e62b1eb-3219-4f2a-b750-63d5179ea18d None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:22.625693 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8558bb08-4c37-4ee2-965d-188eb77f1d58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:22.626399 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8558bb08-4c37-4ee2-965d-188eb77f1d58 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:22.626686 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8558bb08-4c37-4ee2-965d-188eb77f1d58 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:22.626962 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8558bb08-4c37-4ee2-965d-188eb77f1d58 None admin] PATH_INFO: `/v3/users/d5e7e3bd287e431eba21ba5fa1bb0587` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:22.630733 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8558bb08-4c37-4ee2-965d-188eb77f1d58 None admin] RBAC: Authorizing `identity:get_user(user_id=d5e7e3bd287e431eba21ba5fa1bb0587)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:22.632339 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8558bb08-4c37-4ee2-965d-188eb77f1d58 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:22.634087 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 142/291] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Aug 24 17:27:22 2022] GET /identity/v3/users/d5e7e3bd287e431eba21ba5fa1bb0587 => generated 297 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:27:22.640081 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724306ff-f826-48d0-941d-7415629435df None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:22.753978 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724306ff-f826-48d0-941d-7415629435df None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:22.779511 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:22.803565 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28936b12-2cd0-4ae5-b1eb-2f9260e6af04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:22.804431 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28936b12-2cd0-4ae5-b1eb-2f9260e6af04 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:22.804942 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28936b12-2cd0-4ae5-b1eb-2f9260e6af04 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:22.805413 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28936b12-2cd0-4ae5-b1eb-2f9260e6af04 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:22.881355 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28936b12-2cd0-4ae5-b1eb-2f9260e6af04 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:22.883184 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28936b12-2cd0-4ae5-b1eb-2f9260e6af04 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:23.016151 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 150/292] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Aug 24 17:27:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 17:27:23.022189 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8558bb08-4c37-4ee2-965d-188eb77f1d58 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:23.163388 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8558bb08-4c37-4ee2-965d-188eb77f1d58 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:23.209436 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc784fb-01c6-4248-aadf-003dd4449243 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:23.210078 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc784fb-01c6-4248-aadf-003dd4449243 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:23.210356 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc784fb-01c6-4248-aadf-003dd4449243 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:23.210634 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc784fb-01c6-4248-aadf-003dd4449243 None admin] PATH_INFO: `/v3/projects/3a29643b449b4fc1aa71802d8db7c0d2` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:23.211785 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc784fb-01c6-4248-aadf-003dd4449243 None admin] RBAC: Authorizing `identity:get_project(project_id=3a29643b449b4fc1aa71802d8db7c0d2)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:23.213322 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc784fb-01c6-4248-aadf-003dd4449243 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:23.215273 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 143/293] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Aug 24 17:27:23 2022] GET /identity/v3/projects/3a29643b449b4fc1aa71802d8db7c0d2 => generated 302 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:27:23.221679 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28936b12-2cd0-4ae5-b1eb-2f9260e6af04 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:23.242515 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:23.328879 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28936b12-2cd0-4ae5-b1eb-2f9260e6af04 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:23.363508 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cccca93-3829-4a49-97b4-1bfe883e0f34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:23.364519 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cccca93-3829-4a49-97b4-1bfe883e0f34 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:23.365058 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cccca93-3829-4a49-97b4-1bfe883e0f34 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:23.365555 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cccca93-3829-4a49-97b4-1bfe883e0f34 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:23.367027 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cccca93-3829-4a49-97b4-1bfe883e0f34 None admin] RBAC: Adding query filter params (role.id=02476a5b82eb4b188c18a5d706a3ef68, scope.project.id=3a29643b449b4fc1aa71802d8db7c0d2, user.id=d5e7e3bd287e431eba21ba5fa1bb0587) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:27:23.367603 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cccca93-3829-4a49-97b4-1bfe883e0f34 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:23.370910 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cccca93-3829-4a49-97b4-1bfe883e0f34 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:23.379439 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 151/294] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Aug 24 17:27:23 2022] GET /identity/v3/role_assignments?user.id=d5e7e3bd287e431eba21ba5fa1bb0587&scope.project.id=3a29643b449b4fc1aa71802d8db7c0d2&role.id=02476a5b82eb4b188c18a5d706a3ef68 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:27:23.793845 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c0f7b6-10c5-4b2e-8c18-93a3eb77d7ee None None] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:23.794392 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c0f7b6-10c5-4b2e-8c18-93a3eb77d7ee None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:23.794392 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c0f7b6-10c5-4b2e-8c18-93a3eb77d7ee None None] PATH_INFO: `/` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:23.798114 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 144/295] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:27:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:27:23.803893 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b942ae7-c655-483e-aac6-5d40ed9c76d6 None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:23.804189 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b942ae7-c655-483e-aac6-5d40ed9c76d6 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:23.804475 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b942ae7-c655-483e-aac6-5d40ed9c76d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:23.830318 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-2b942ae7-c655-483e-aac6-5d40ed9c76d6 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:27:23.831520 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-2b942ae7-c655-483e-aac6-5d40ed9c76d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:27:23.906635 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 152/296] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:27:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:27:24.051498 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d97db52-f7f7-454c-a92a-41a734813ba9 None None] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:24.051687 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d97db52-f7f7-454c-a92a-41a734813ba9 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:24.051970 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d97db52-f7f7-454c-a92a-41a734813ba9 None None] PATH_INFO: `/` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:24.053628 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 145/297] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:27:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:27:24.058830 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6607980d-01f5-4130-87bc-8212281c7b99 None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:24.059120 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6607980d-01f5-4130-87bc-8212281c7b99 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:24.059397 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6607980d-01f5-4130-87bc-8212281c7b99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:24.077366 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-6607980d-01f5-4130-87bc-8212281c7b99 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:27:24.078556 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-6607980d-01f5-4130-87bc-8212281c7b99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:27:24.086168 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:24.150002 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 153/298] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:27:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:27:24.203555 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d97db52-f7f7-454c-a92a-41a734813ba9 None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:24.216843 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:24.247133 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abf72b3-0bb9-488c-a23e-93401e60df01 None None] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:24.247640 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abf72b3-0bb9-488c-a23e-93401e60df01 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:24.247713 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abf72b3-0bb9-488c-a23e-93401e60df01 None None] PATH_INFO: `/` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:24.249319 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 154/299] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:27:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 17:27:24.254105 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85441dee-e9cb-46e2-94c3-da974a6154eb None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:24.254394 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85441dee-e9cb-46e2-94c3-da974a6154eb None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:24.254670 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85441dee-e9cb-46e2-94c3-da974a6154eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:24.272295 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-85441dee-e9cb-46e2-94c3-da974a6154eb None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:27:24.273623 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-85441dee-e9cb-46e2-94c3-da974a6154eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:27:24.321131 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d97db52-f7f7-454c-a92a-41a734813ba9 None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:24.357724 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c95c02a-f2cd-479c-8683-404cfc822438 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:24.358360 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c95c02a-f2cd-479c-8683-404cfc822438 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:24.358640 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c95c02a-f2cd-479c-8683-404cfc822438 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:24.358917 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c95c02a-f2cd-479c-8683-404cfc822438 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:24.359229 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 155/300] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:27:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 17:27:24.372294 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c95c02a-f2cd-479c-8683-404cfc822438 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:27:24.377329 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application [None req-0c95c02a-f2cd-479c-8683-404cfc822438 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 17:27:24.377329 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:27:24.377329 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:27:24.377329 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:27:24.377329 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:27:24.377329 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:27:24.377329 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:27:24.377329 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:27:24.377329 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:27:24.377329 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:27:24.377329 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:27:24.377329 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:27:24.377329 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:27:24.377329 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:27:24.377329 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:27:24.377329 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:27:24.378169 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:27:24.378169 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:27:24.378169 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:27:24.378169 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:27:24.378169 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:27:24.378169 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:27:24.378169 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:27:24.378169 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:27:24.378169 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:27:24.378169 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:27:24.378169 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:27:24.378169 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application with Lock( Aug 24 17:27:24.378169 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:27:24.378169 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:27:24.378169 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:27:24.378169 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:27:24.378954 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:27:24.378954 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:27:24.378954 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:27:24.378954 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:27:24.378954 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:27:24.378954 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:27:24.378954 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:27:24.378954 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:27:24.378954 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:27:24.378954 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:27:24.378954 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 17:27:24.378954 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Aug 24 17:27:24.384650 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 146/301] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Aug 24 17:27:24 2022] GET /identity/v3/roles/reader => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:27:24.387556 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85441dee-e9cb-46e2-94c3-da974a6154eb None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:24.434887 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c95c02a-f2cd-479c-8683-404cfc822438 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:24.464849 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:24.497788 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85441dee-e9cb-46e2-94c3-da974a6154eb None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:24.530257 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:24.539064 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbf7e5a-7cfa-49a6-afc7-8d059eb17710 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:24.539886 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbf7e5a-7cfa-49a6-afc7-8d059eb17710 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:24.540331 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbf7e5a-7cfa-49a6-afc7-8d059eb17710 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:24.541226 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbf7e5a-7cfa-49a6-afc7-8d059eb17710 None admin] PATH_INFO: `/v3/roles` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:24.542583 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbf7e5a-7cfa-49a6-afc7-8d059eb17710 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:27:24.543444 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbf7e5a-7cfa-49a6-afc7-8d059eb17710 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:24.545280 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbf7e5a-7cfa-49a6-afc7-8d059eb17710 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:24.562383 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 156/302] 10.222.0.45 () {56 vars in 1101 bytes} [Wed Aug 24 17:27:24 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:27:24.564321 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbf7e5a-7cfa-49a6-afc7-8d059eb17710 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:24.571790 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c95c02a-f2cd-479c-8683-404cfc822438 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:24.615582 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0809af40-8214-432f-8e40-da025d2cabaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:24.616789 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0809af40-8214-432f-8e40-da025d2cabaf None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:24.617564 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0809af40-8214-432f-8e40-da025d2cabaf None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:24.618032 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0809af40-8214-432f-8e40-da025d2cabaf None admin] PATH_INFO: `/v3/domains/default` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:24.619350 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0809af40-8214-432f-8e40-da025d2cabaf None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:24.621885 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0809af40-8214-432f-8e40-da025d2cabaf None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:24.624772 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 147/303] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 17:27:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:27:24.630651 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0809af40-8214-432f-8e40-da025d2cabaf None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:24.678287 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbf7e5a-7cfa-49a6-afc7-8d059eb17710 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:24.712783 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cebbcb-b57e-4f74-bbb9-dfeeb8535c88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:24.713449 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cebbcb-b57e-4f74-bbb9-dfeeb8535c88 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:24.713731 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cebbcb-b57e-4f74-bbb9-dfeeb8535c88 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:24.714005 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cebbcb-b57e-4f74-bbb9-dfeeb8535c88 None admin] PATH_INFO: `/v3/users/15ce3a91b6d84377941e9c83e617bd46` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:24.715879 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:24.727877 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cebbcb-b57e-4f74-bbb9-dfeeb8535c88 None admin] RBAC: Authorizing `identity:get_user(user_id=15ce3a91b6d84377941e9c83e617bd46)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:24.730425 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cebbcb-b57e-4f74-bbb9-dfeeb8535c88 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:24.732060 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 157/304] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Aug 24 17:27:24 2022] GET /identity/v3/users/15ce3a91b6d84377941e9c83e617bd46 => generated 293 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:27:24.732913 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cebbcb-b57e-4f74-bbb9-dfeeb8535c88 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:24.733606 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0809af40-8214-432f-8e40-da025d2cabaf None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:24.767547 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22edd0a-0c09-46f6-8766-23ee38ca702c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:24.768165 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22edd0a-0c09-46f6-8766-23ee38ca702c None admin] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:24.768447 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22edd0a-0c09-46f6-8766-23ee38ca702c None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:24.768750 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22edd0a-0c09-46f6-8766-23ee38ca702c None admin] PATH_INFO: `/v3/groups` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:24.769386 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22edd0a-0c09-46f6-8766-23ee38ca702c None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:24.770880 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22edd0a-0c09-46f6-8766-23ee38ca702c None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:24.783108 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:24.788592 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 148/305] 10.222.0.45 () {60 vars in 1134 bytes} [Wed Aug 24 17:27:24 2022] POST /identity/v3/groups => generated 229 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:27:24.789647 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0028de82-8a1a-4bd6-8a2b-383d78d4ae7f None None] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:24.790395 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0028de82-8a1a-4bd6-8a2b-383d78d4ae7f None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:24.790668 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0028de82-8a1a-4bd6-8a2b-383d78d4ae7f None None] PATH_INFO: `/` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:24.797166 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 149/306] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:27:24 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:27:24.799004 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0028de82-8a1a-4bd6-8a2b-383d78d4ae7f None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:24.851337 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cebbcb-b57e-4f74-bbb9-dfeeb8535c88 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:24.895630 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d949ffbe-ba4f-4b13-9100-a381f66d5a35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:24.896287 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d949ffbe-ba4f-4b13-9100-a381f66d5a35 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:24.896594 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d949ffbe-ba4f-4b13-9100-a381f66d5a35 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:24.896872 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d949ffbe-ba4f-4b13-9100-a381f66d5a35 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:24.909969 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d949ffbe-ba4f-4b13-9100-a381f66d5a35 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:27:24.913325 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0028de82-8a1a-4bd6-8a2b-383d78d4ae7f None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:24.915257 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application [None req-d949ffbe-ba4f-4b13-9100-a381f66d5a35 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 17:27:24.915257 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:27:24.915257 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:27:24.915257 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:27:24.915257 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:27:24.915257 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:27:24.915257 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:27:24.915257 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:27:24.915257 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:27:24.915257 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:27:24.915257 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:27:24.915257 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:27:24.915257 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:27:24.915257 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:27:24.915257 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:27:24.915257 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:27:24.916147 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:27:24.916147 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:27:24.916147 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:27:24.916147 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:27:24.916147 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:27:24.916147 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:27:24.916147 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:27:24.916147 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:27:24.916147 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:27:24.916147 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:27:24.916147 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:27:24.916147 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application with Lock( Aug 24 17:27:24.916147 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:27:24.916147 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:27:24.916147 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:27:24.916147 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:27:24.917617 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:27:24.917617 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:27:24.917617 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:27:24.917617 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:27:24.917617 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:27:24.917617 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:27:24.917617 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:27:24.917617 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:27:24.917617 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:27:24.917617 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:27:24.917617 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 17:27:24.917617 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Aug 24 17:27:24.917617 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 158/307] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Aug 24 17:27:24 2022] GET /identity/v3/roles/member => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:27:24.923200 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde599e7-be39-4be3-9d2e-86e3f5347b8d None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:24.923488 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde599e7-be39-4be3-9d2e-86e3f5347b8d None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:24.923765 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde599e7-be39-4be3-9d2e-86e3f5347b8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:24.942398 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-bde599e7-be39-4be3-9d2e-86e3f5347b8d None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:27:24.944154 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-bde599e7-be39-4be3-9d2e-86e3f5347b8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:27:24.959499 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d869739-b175-4092-b617-2a729505286c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:24.960457 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d869739-b175-4092-b617-2a729505286c None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:24.960783 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d869739-b175-4092-b617-2a729505286c None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:24.961063 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d869739-b175-4092-b617-2a729505286c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:24.962021 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d869739-b175-4092-b617-2a729505286c None admin] RBAC: Adding query filter params (role.id=645db6d227604357b1949889630755ff, user.id=15ce3a91b6d84377941e9c83e617bd46, scope.system=all) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:27:24.963040 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d869739-b175-4092-b617-2a729505286c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:24.965567 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d869739-b175-4092-b617-2a729505286c None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:24.971857 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 150/308] 10.222.0.45 () {56 vars in 1308 bytes} [Wed Aug 24 17:27:24 2022] GET /identity/v3/role_assignments?user.id=15ce3a91b6d84377941e9c83e617bd46&scope.system=all&role.id=645db6d227604357b1949889630755ff => generated 226 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:27:24.972317 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d869739-b175-4092-b617-2a729505286c None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:24.997119 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:25.026899 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 159/309] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:27:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:27:25.058196 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:25.105523 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d869739-b175-4092-b617-2a729505286c None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:25.143908 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff722a4f-eb83-4b60-9486-82c2dff7012a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:25.144571 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff722a4f-eb83-4b60-9486-82c2dff7012a None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:25.144860 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff722a4f-eb83-4b60-9486-82c2dff7012a None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:25.145130 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff722a4f-eb83-4b60-9486-82c2dff7012a None admin] PATH_INFO: `/v3/roles` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:25.145798 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff722a4f-eb83-4b60-9486-82c2dff7012a None admin] RBAC: Adding query filter params (name=member) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:27:25.146097 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff722a4f-eb83-4b60-9486-82c2dff7012a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:25.147467 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff722a4f-eb83-4b60-9486-82c2dff7012a None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:25.156703 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 151/310] 10.222.0.45 () {56 vars in 1101 bytes} [Wed Aug 24 17:27:24 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:27:25.161884 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde599e7-be39-4be3-9d2e-86e3f5347b8d None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:25.267291 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde599e7-be39-4be3-9d2e-86e3f5347b8d None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:25.302677 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2eb2bd-6f9d-4cca-954a-a13d4ee9ae68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:25.303323 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2eb2bd-6f9d-4cca-954a-a13d4ee9ae68 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:25.303604 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2eb2bd-6f9d-4cca-954a-a13d4ee9ae68 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:25.303883 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2eb2bd-6f9d-4cca-954a-a13d4ee9ae68 None admin] PATH_INFO: `/v3/users/a864394f741741a49f3d25e30f38c231` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:25.305054 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2eb2bd-6f9d-4cca-954a-a13d4ee9ae68 None admin] RBAC: Authorizing `identity:get_user(user_id=a864394f741741a49f3d25e30f38c231)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:25.306584 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2eb2bd-6f9d-4cca-954a-a13d4ee9ae68 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:25.307919 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff722a4f-eb83-4b60-9486-82c2dff7012a None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:25.309060 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 160/311] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Aug 24 17:27:25 2022] GET /identity/v3/users/a864394f741741a49f3d25e30f38c231 => generated 293 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:27:25.314644 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2eb2bd-6f9d-4cca-954a-a13d4ee9ae68 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:25.366335 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:25.419063 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2eb2bd-6f9d-4cca-954a-a13d4ee9ae68 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:25.421526 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff722a4f-eb83-4b60-9486-82c2dff7012a None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:25.446786 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:25.458152 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff9905e-400b-4566-8644-0282f51fa59a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:25.458810 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff9905e-400b-4566-8644-0282f51fa59a None admin] REQUEST_METHOD: `PUT` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:25.459092 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff9905e-400b-4566-8644-0282f51fa59a None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:25.459367 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff9905e-400b-4566-8644-0282f51fa59a None admin] PATH_INFO: `/v3/system/users/a864394f741741a49f3d25e30f38c231/roles/02476a5b82eb4b188c18a5d706a3ef68` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:25.460822 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff9905e-400b-4566-8644-0282f51fa59a None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=a864394f741741a49f3d25e30f38c231, role_id=02476a5b82eb4b188c18a5d706a3ef68)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:25.461131 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b3ffc5-6816-4b12-bf8e-cad5581c3fd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:25.461963 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b3ffc5-6816-4b12-bf8e-cad5581c3fd0 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:25.462197 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff9905e-400b-4566-8644-0282f51fa59a None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:25.462624 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b3ffc5-6816-4b12-bf8e-cad5581c3fd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:25.463079 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b3ffc5-6816-4b12-bf8e-cad5581c3fd0 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:25.476641 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 161/312] 10.222.0.45 () {58 vars in 1334 bytes} [Wed Aug 24 17:27:25 2022] PUT /identity/v3/system/users/a864394f741741a49f3d25e30f38c231/roles/02476a5b82eb4b188c18a5d706a3ef68 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:27:25.483027 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b3ffc5-6816-4b12-bf8e-cad5581c3fd0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:27:25.489163 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application [None req-97b3ffc5-6816-4b12-bf8e-cad5581c3fd0 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 17:27:25.489163 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:27:25.489163 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:27:25.489163 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:27:25.489163 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:27:25.489163 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:27:25.489163 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:27:25.489163 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:27:25.489163 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:27:25.489163 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:27:25.489163 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:27:25.489163 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:27:25.489163 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:27:25.489163 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:27:25.489163 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:27:25.489163 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:27:25.490136 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:27:25.490136 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:27:25.490136 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:27:25.490136 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:27:25.490136 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:27:25.490136 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:27:25.490136 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:27:25.490136 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:27:25.490136 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:27:25.490136 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:27:25.490136 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:27:25.490136 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application with Lock( Aug 24 17:27:25.490136 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:27:25.490136 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:27:25.490136 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:27:25.490136 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:27:25.490915 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:27:25.490915 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:27:25.490915 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:27:25.490915 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:27:25.490915 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:27:25.490915 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:27:25.490915 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:27:25.490915 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:27:25.490915 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:27:25.490915 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:27:25.490915 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 17:27:25.490915 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Aug 24 17:27:25.500015 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 152/313] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Aug 24 17:27:25 2022] GET /identity/v3/roles/reader => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:27:25.501919 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff9905e-400b-4566-8644-0282f51fa59a None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:25.621132 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff9905e-400b-4566-8644-0282f51fa59a None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:25.636766 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:25.660613 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36175f29-7f86-4ccc-9d63-f3fc8b2203b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:25.661225 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36175f29-7f86-4ccc-9d63-f3fc8b2203b8 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:25.661521 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36175f29-7f86-4ccc-9d63-f3fc8b2203b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:25.661774 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36175f29-7f86-4ccc-9d63-f3fc8b2203b8 None admin] PATH_INFO: `/v3/roles` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:25.662439 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36175f29-7f86-4ccc-9d63-f3fc8b2203b8 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:27:25.662749 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36175f29-7f86-4ccc-9d63-f3fc8b2203b8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:25.664128 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36175f29-7f86-4ccc-9d63-f3fc8b2203b8 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:25.677763 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 162/314] 10.222.0.45 () {56 vars in 1101 bytes} [Wed Aug 24 17:27:25 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:27:25.679135 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b3ffc5-6816-4b12-bf8e-cad5581c3fd0 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:25.791310 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b3ffc5-6816-4b12-bf8e-cad5581c3fd0 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:25.827191 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbcedf2-d3fb-4a09-acba-082951b71417 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:25.827868 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbcedf2-d3fb-4a09-acba-082951b71417 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:25.828131 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbcedf2-d3fb-4a09-acba-082951b71417 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:25.828367 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbcedf2-d3fb-4a09-acba-082951b71417 None admin] PATH_INFO: `/v3/users/2ee480620a7d40ad923f7d6e823d7b05` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:25.829557 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbcedf2-d3fb-4a09-acba-082951b71417 None admin] RBAC: Authorizing `identity:get_user(user_id=2ee480620a7d40ad923f7d6e823d7b05)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:25.831011 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbcedf2-d3fb-4a09-acba-082951b71417 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:25.832924 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 153/315] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Aug 24 17:27:25 2022] GET /identity/v3/users/2ee480620a7d40ad923f7d6e823d7b05 => generated 297 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:27:25.839363 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36175f29-7f86-4ccc-9d63-f3fc8b2203b8 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:25.951218 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36175f29-7f86-4ccc-9d63-f3fc8b2203b8 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:25.988288 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a549fa80-53c0-4068-945b-a8a4111b9d3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:25.988956 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a549fa80-53c0-4068-945b-a8a4111b9d3d None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:25.989238 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a549fa80-53c0-4068-945b-a8a4111b9d3d None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:25.989512 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a549fa80-53c0-4068-945b-a8a4111b9d3d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:26.036225 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:26.061188 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a549fa80-53c0-4068-945b-a8a4111b9d3d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:26.062700 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a549fa80-53c0-4068-945b-a8a4111b9d3d None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:26.166350 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 163/316] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Aug 24 17:27:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 17:27:26.172646 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbcedf2-d3fb-4a09-acba-082951b71417 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:26.228830 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:26.282656 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbcedf2-d3fb-4a09-acba-082951b71417 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:26.316796 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9251cbe4-9508-441d-9b22-bc4b7cf02e4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:26.317428 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9251cbe4-9508-441d-9b22-bc4b7cf02e4d None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:26.317708 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9251cbe4-9508-441d-9b22-bc4b7cf02e4d None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:26.318045 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9251cbe4-9508-441d-9b22-bc4b7cf02e4d None admin] PATH_INFO: `/v3/projects/3a29643b449b4fc1aa71802d8db7c0d2` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:26.319219 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9251cbe4-9508-441d-9b22-bc4b7cf02e4d None admin] RBAC: Authorizing `identity:get_project(project_id=3a29643b449b4fc1aa71802d8db7c0d2)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:26.320689 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9251cbe4-9508-441d-9b22-bc4b7cf02e4d None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:26.322667 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 154/317] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Aug 24 17:27:26 2022] GET /identity/v3/projects/3a29643b449b4fc1aa71802d8db7c0d2 => generated 302 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:27:26.329047 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a549fa80-53c0-4068-945b-a8a4111b9d3d None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:26.407633 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:26.433146 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a549fa80-53c0-4068-945b-a8a4111b9d3d None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:26.468661 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d910a0a5-ad65-4eed-a39e-26e94baf2a96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:26.469275 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d910a0a5-ad65-4eed-a39e-26e94baf2a96 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:26.469557 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d910a0a5-ad65-4eed-a39e-26e94baf2a96 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:26.469841 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d910a0a5-ad65-4eed-a39e-26e94baf2a96 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:26.470575 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d910a0a5-ad65-4eed-a39e-26e94baf2a96 None admin] RBAC: Adding query filter params (role.id=645db6d227604357b1949889630755ff, scope.project.id=3a29643b449b4fc1aa71802d8db7c0d2, user.id=2ee480620a7d40ad923f7d6e823d7b05) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:27:26.470885 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d910a0a5-ad65-4eed-a39e-26e94baf2a96 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:26.472330 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d910a0a5-ad65-4eed-a39e-26e94baf2a96 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:26.479499 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 164/318] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Aug 24 17:27:26 2022] GET /identity/v3/role_assignments?user.id=2ee480620a7d40ad923f7d6e823d7b05&scope.project.id=3a29643b449b4fc1aa71802d8db7c0d2&role.id=645db6d227604357b1949889630755ff => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:27:26.887801 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36902978-d396-48f6-a345-cb2b4fb0567d None None] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:26.888325 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36902978-d396-48f6-a345-cb2b4fb0567d None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:26.888325 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36902978-d396-48f6-a345-cb2b4fb0567d None None] PATH_INFO: `/` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:26.889869 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 155/319] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:27:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:27:26.894882 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0064d8ef-6598-4749-858d-63cf908a46ad None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:26.895173 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0064d8ef-6598-4749-858d-63cf908a46ad None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:26.895453 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0064d8ef-6598-4749-858d-63cf908a46ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:26.913748 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-0064d8ef-6598-4749-858d-63cf908a46ad None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:27:26.914946 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-0064d8ef-6598-4749-858d-63cf908a46ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:27:26.944424 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e119c771-a0cb-40ad-9d26-a60367d40f9f None None] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:26.944739 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e119c771-a0cb-40ad-9d26-a60367d40f9f None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:26.945018 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e119c771-a0cb-40ad-9d26-a60367d40f9f None None] PATH_INFO: `/` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:26.946357 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 156/320] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:27:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 17:27:26.951815 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0932b077-61d0-410f-b95a-f91a861673e5 None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:26.952107 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0932b077-61d0-410f-b95a-f91a861673e5 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:26.952761 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0932b077-61d0-410f-b95a-f91a861673e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:26.970651 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-0932b077-61d0-410f-b95a-f91a861673e5 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:27:26.971841 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-0932b077-61d0-410f-b95a-f91a861673e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:27:26.986139 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 165/321] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:27:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:27:27.038166 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 157/322] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:27:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 17:27:27.255010 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0064d8ef-6598-4749-858d-63cf908a46ad None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:27.300226 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0932b077-61d0-410f-b95a-f91a861673e5 None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:27.326989 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:27.356466 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:27.360742 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0064d8ef-6598-4749-858d-63cf908a46ad None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:27.396244 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee8a4c1-4bde-4690-971d-27319f25d5c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:27.396906 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee8a4c1-4bde-4690-971d-27319f25d5c8 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:27.397197 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee8a4c1-4bde-4690-971d-27319f25d5c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:27.397472 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee8a4c1-4bde-4690-971d-27319f25d5c8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:27.398585 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee8a4c1-4bde-4690-971d-27319f25d5c8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:27.400031 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee8a4c1-4bde-4690-971d-27319f25d5c8 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:27.402396 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 166/323] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 17:27:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:27:27.407544 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee8a4c1-4bde-4690-971d-27319f25d5c8 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:27.408193 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0932b077-61d0-410f-b95a-f91a861673e5 None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:27.444512 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4421bf7e-42c0-4c4b-88b1-d423c149424d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:27.445164 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4421bf7e-42c0-4c4b-88b1-d423c149424d None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:27.445443 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4421bf7e-42c0-4c4b-88b1-d423c149424d None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:27.445711 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4421bf7e-42c0-4c4b-88b1-d423c149424d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:27.457040 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4421bf7e-42c0-4c4b-88b1-d423c149424d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:27:27.461747 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application [None req-4421bf7e-42c0-4c4b-88b1-d423c149424d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 17:27:27.461747 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:27:27.461747 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:27:27.461747 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:27:27.461747 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:27:27.461747 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:27:27.461747 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:27:27.461747 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:27:27.461747 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:27:27.461747 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:27:27.461747 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:27:27.461747 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:27:27.461747 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:27:27.461747 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:27:27.461747 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:27:27.461747 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:27:27.462628 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:27:27.462628 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:27:27.462628 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:27:27.462628 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:27:27.462628 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:27:27.462628 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:27:27.462628 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:27:27.462628 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:27:27.462628 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:27:27.462628 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:27:27.462628 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:27:27.462628 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application with Lock( Aug 24 17:27:27.462628 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:27:27.462628 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:27:27.462628 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:27:27.462628 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:27:27.463994 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:27:27.463994 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:27:27.463994 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:27:27.463994 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:27:27.463994 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:27:27.463994 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:27:27.463994 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:27:27.463994 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:27:27.463994 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:27:27.463994 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:27:27.463994 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 17:27:27.463994 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Aug 24 17:27:27.463994 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 158/324] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Aug 24 17:27:27 2022] GET /identity/v3/roles/reader => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 17:27:27.464804 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892d70a9-b9b0-4b95-9832-ca9347c49d92 None None] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:27.464869 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892d70a9-b9b0-4b95-9832-ca9347c49d92 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:27.465122 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892d70a9-b9b0-4b95-9832-ca9347c49d92 None None] PATH_INFO: `/` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:27.466363 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 159/325] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:27:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:27:27.469780 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892d70a9-b9b0-4b95-9832-ca9347c49d92 None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:27.511136 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee8a4c1-4bde-4690-971d-27319f25d5c8 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:27.543783 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5407502e-79a1-4ee8-b9fd-077a4c963399 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:27.544390 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5407502e-79a1-4ee8-b9fd-077a4c963399 None admin] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:27.544697 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5407502e-79a1-4ee8-b9fd-077a4c963399 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:27.544973 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5407502e-79a1-4ee8-b9fd-077a4c963399 None admin] PATH_INFO: `/v3/groups` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:27.545597 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5407502e-79a1-4ee8-b9fd-077a4c963399 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:27.547038 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5407502e-79a1-4ee8-b9fd-077a4c963399 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:27.557877 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:27.563937 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 167/326] 10.222.0.45 () {60 vars in 1134 bytes} [Wed Aug 24 17:27:27 2022] POST /identity/v3/groups => generated 226 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:27:27.565141 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0033c57c-973d-4dde-9a02-98fdb4be8835 None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:27.565774 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0033c57c-973d-4dde-9a02-98fdb4be8835 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:27.566140 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0033c57c-973d-4dde-9a02-98fdb4be8835 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:27.578159 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892d70a9-b9b0-4b95-9832-ca9347c49d92 None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:27.586128 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-0033c57c-973d-4dde-9a02-98fdb4be8835 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:27:27.587378 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-0033c57c-973d-4dde-9a02-98fdb4be8835 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:27:27.593856 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:27.622188 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5675ec1b-1090-41f2-bc05-f833614e00b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:27.622820 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5675ec1b-1090-41f2-bc05-f833614e00b2 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:27.623203 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5675ec1b-1090-41f2-bc05-f833614e00b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:27.623482 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5675ec1b-1090-41f2-bc05-f833614e00b2 None admin] PATH_INFO: `/v3/roles` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:27.624167 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5675ec1b-1090-41f2-bc05-f833614e00b2 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:27:27.624604 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5675ec1b-1090-41f2-bc05-f833614e00b2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:27.626299 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5675ec1b-1090-41f2-bc05-f833614e00b2 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:27.636409 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 160/327] 10.222.0.45 () {56 vars in 1101 bytes} [Wed Aug 24 17:27:27 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:27:27.643016 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5675ec1b-1090-41f2-bc05-f833614e00b2 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:27.666557 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 168/328] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:27:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:27:27.756070 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5675ec1b-1090-41f2-bc05-f833614e00b2 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:27.793575 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9f2429-9770-4b7c-8ab5-047ce45a9ec2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:27.794901 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9f2429-9770-4b7c-8ab5-047ce45a9ec2 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:27.794901 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9f2429-9770-4b7c-8ab5-047ce45a9ec2 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:27.794901 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9f2429-9770-4b7c-8ab5-047ce45a9ec2 None admin] PATH_INFO: `/v3/users/15ce3a91b6d84377941e9c83e617bd46` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:27.795901 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9f2429-9770-4b7c-8ab5-047ce45a9ec2 None admin] RBAC: Authorizing `identity:get_user(user_id=15ce3a91b6d84377941e9c83e617bd46)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:27.798824 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9f2429-9770-4b7c-8ab5-047ce45a9ec2 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:27.799567 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 161/329] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Aug 24 17:27:27 2022] GET /identity/v3/users/15ce3a91b6d84377941e9c83e617bd46 => generated 293 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:27:27.805416 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0033c57c-973d-4dde-9a02-98fdb4be8835 None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:27.915699 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0033c57c-973d-4dde-9a02-98fdb4be8835 None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:27.931381 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9f2429-9770-4b7c-8ab5-047ce45a9ec2 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:27.946202 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:27.956896 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557ee3e9-91e0-421c-bb1c-946c6d420642 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:27.957558 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557ee3e9-91e0-421c-bb1c-946c6d420642 None admin] REQUEST_METHOD: `PUT` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:27.957838 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557ee3e9-91e0-421c-bb1c-946c6d420642 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:27.958113 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557ee3e9-91e0-421c-bb1c-946c6d420642 None admin] PATH_INFO: `/v3/system/users/15ce3a91b6d84377941e9c83e617bd46/roles/645db6d227604357b1949889630755ff` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:27.959989 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557ee3e9-91e0-421c-bb1c-946c6d420642 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=15ce3a91b6d84377941e9c83e617bd46, role_id=645db6d227604357b1949889630755ff)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:27.961670 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557ee3e9-91e0-421c-bb1c-946c6d420642 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:27.976714 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 169/330] 10.222.0.45 () {58 vars in 1334 bytes} [Wed Aug 24 17:27:27 2022] PUT /identity/v3/system/users/15ce3a91b6d84377941e9c83e617bd46/roles/645db6d227604357b1949889630755ff => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:27:27.990089 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:28.063710 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9f2429-9770-4b7c-8ab5-047ce45a9ec2 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:28.099956 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3a6022-e379-43a6-b292-ce42db12dc6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:28.100628 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3a6022-e379-43a6-b292-ce42db12dc6a None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:28.100907 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3a6022-e379-43a6-b292-ce42db12dc6a None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:28.101174 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3a6022-e379-43a6-b292-ce42db12dc6a None admin] PATH_INFO: `/v3/roles/member` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:28.113900 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3a6022-e379-43a6-b292-ce42db12dc6a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:27:28.119462 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application [None req-de3a6022-e379-43a6-b292-ce42db12dc6a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 17:27:28.119462 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:27:28.119462 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:27:28.119462 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:27:28.119462 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:27:28.119462 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:27:28.119462 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:27:28.119462 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:27:28.119462 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:27:28.119462 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:27:28.119462 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:27:28.119462 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:27:28.119462 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:27:28.119462 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:27:28.119462 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:27:28.119462 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:27:28.120972 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:27:28.120972 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:27:28.120972 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:27:28.120972 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:27:28.120972 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:27:28.120972 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:27:28.120972 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:27:28.120972 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:27:28.120972 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:27:28.120972 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:27:28.120972 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:27:28.120972 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application with Lock( Aug 24 17:27:28.120972 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:27:28.120972 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:27:28.120972 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:27:28.120972 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:27:28.121747 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:27:28.121747 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:27:28.121747 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:27:28.121747 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:27:28.121747 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:27:28.121747 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:27:28.121747 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:27:28.121747 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:27:28.121747 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:27:28.121747 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:27:28.121747 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 17:27:28.121747 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Aug 24 17:27:28.121747 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 162/331] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Aug 24 17:27:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:27:28.127625 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557ee3e9-91e0-421c-bb1c-946c6d420642 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:28.244608 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557ee3e9-91e0-421c-bb1c-946c6d420642 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:28.279604 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe30e41-4c0c-45d4-b594-d71d19152140 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:28.280216 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe30e41-4c0c-45d4-b594-d71d19152140 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:28.280493 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe30e41-4c0c-45d4-b594-d71d19152140 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:28.280792 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe30e41-4c0c-45d4-b594-d71d19152140 None admin] PATH_INFO: `/v3/roles` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:28.281460 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe30e41-4c0c-45d4-b594-d71d19152140 None admin] RBAC: Adding query filter params (name=member) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:27:28.281761 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe30e41-4c0c-45d4-b594-d71d19152140 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:28.283137 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe30e41-4c0c-45d4-b594-d71d19152140 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:28.294823 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 170/332] 10.222.0.45 () {56 vars in 1101 bytes} [Wed Aug 24 17:27:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:27:28.300347 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3a6022-e379-43a6-b292-ce42db12dc6a None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:28.416150 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:28.423522 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3a6022-e379-43a6-b292-ce42db12dc6a None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:28.464378 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4ee843-476d-4213-a351-dfade7e1aca2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:28.465061 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4ee843-476d-4213-a351-dfade7e1aca2 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:28.465337 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4ee843-476d-4213-a351-dfade7e1aca2 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:28.465609 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4ee843-476d-4213-a351-dfade7e1aca2 None admin] PATH_INFO: `/v3/users/a864394f741741a49f3d25e30f38c231` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:28.468266 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4ee843-476d-4213-a351-dfade7e1aca2 None admin] RBAC: Authorizing `identity:get_user(user_id=a864394f741741a49f3d25e30f38c231)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:28.469769 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4ee843-476d-4213-a351-dfade7e1aca2 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:28.471353 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 163/333] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Aug 24 17:27:28 2022] GET /identity/v3/users/a864394f741741a49f3d25e30f38c231 => generated 293 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:27:28.477367 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe30e41-4c0c-45d4-b594-d71d19152140 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:28.540767 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:28.594172 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe30e41-4c0c-45d4-b594-d71d19152140 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:28.630743 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f6037c-e1ee-46b2-a1f1-cfd1f990c32b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:28.631356 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f6037c-e1ee-46b2-a1f1-cfd1f990c32b None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:28.631632 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f6037c-e1ee-46b2-a1f1-cfd1f990c32b None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:28.631900 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f6037c-e1ee-46b2-a1f1-cfd1f990c32b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:28.632797 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f6037c-e1ee-46b2-a1f1-cfd1f990c32b None admin] RBAC: Adding query filter params (role.id=02476a5b82eb4b188c18a5d706a3ef68, user.id=a864394f741741a49f3d25e30f38c231, scope.system=all) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:27:28.633108 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f6037c-e1ee-46b2-a1f1-cfd1f990c32b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:28.634596 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f6037c-e1ee-46b2-a1f1-cfd1f990c32b None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:28.652312 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 171/334] 10.222.0.45 () {56 vars in 1308 bytes} [Wed Aug 24 17:27:28 2022] GET /identity/v3/role_assignments?user.id=a864394f741741a49f3d25e30f38c231&scope.system=all&role.id=02476a5b82eb4b188c18a5d706a3ef68 => generated 510 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:27:29.808566 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa95a510-6fd7-45de-b2e1-c5b5ed44db71 None None] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:29.809023 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa95a510-6fd7-45de-b2e1-c5b5ed44db71 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:29.809135 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa95a510-6fd7-45de-b2e1-c5b5ed44db71 None None] PATH_INFO: `/` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:29.810888 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 164/335] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:27:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:27:29.816743 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e814a667-43aa-40cf-be50-29a7cbd06dbc None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:29.817141 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e814a667-43aa-40cf-be50-29a7cbd06dbc None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:29.817307 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e814a667-43aa-40cf-be50-29a7cbd06dbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:29.840422 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-e814a667-43aa-40cf-be50-29a7cbd06dbc None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:27:29.841692 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-e814a667-43aa-40cf-be50-29a7cbd06dbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:27:29.935221 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 172/336] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:27:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:27:30.099099 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dab3335-7ccf-4338-8328-99b0b7fc0e1b None None] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:30.099391 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dab3335-7ccf-4338-8328-99b0b7fc0e1b None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:30.099578 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dab3335-7ccf-4338-8328-99b0b7fc0e1b None None] PATH_INFO: `/` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:30.101207 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 165/337] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:27:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:27:30.107337 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbe9c9a-3098-4294-9ee3-9a9b83976f11 None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:30.107630 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbe9c9a-3098-4294-9ee3-9a9b83976f11 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:30.107907 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbe9c9a-3098-4294-9ee3-9a9b83976f11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:30.131475 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-6cbe9c9a-3098-4294-9ee3-9a9b83976f11 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:27:30.132683 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-6cbe9c9a-3098-4294-9ee3-9a9b83976f11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:27:30.171879 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:30.204855 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 173/338] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:27:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:27:30.252502 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dab3335-7ccf-4338-8328-99b0b7fc0e1b None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:30.371816 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dab3335-7ccf-4338-8328-99b0b7fc0e1b None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:30.414368 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9645b142-1819-4f7e-a7d5-90a801a9fc8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:30.415010 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9645b142-1819-4f7e-a7d5-90a801a9fc8e None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:30.415289 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9645b142-1819-4f7e-a7d5-90a801a9fc8e None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:30.415563 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9645b142-1819-4f7e-a7d5-90a801a9fc8e None admin] PATH_INFO: `/v3/roles/member` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:30.430589 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9645b142-1819-4f7e-a7d5-90a801a9fc8e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:27:30.435985 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application [None req-9645b142-1819-4f7e-a7d5-90a801a9fc8e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 17:27:30.435985 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:27:30.435985 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:27:30.435985 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:27:30.435985 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:27:30.435985 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:27:30.435985 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:27:30.435985 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:27:30.435985 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:27:30.435985 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:27:30.435985 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:27:30.435985 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:27:30.435985 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:27:30.435985 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:27:30.435985 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:27:30.435985 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:27:30.436886 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:27:30.436886 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:27:30.436886 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:27:30.436886 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:27:30.436886 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:27:30.436886 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:27:30.436886 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:27:30.436886 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:27:30.436886 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:27:30.436886 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:27:30.436886 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:27:30.436886 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application with Lock( Aug 24 17:27:30.436886 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:27:30.436886 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:27:30.436886 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:27:30.436886 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:27:30.437681 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:27:30.437681 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:27:30.437681 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:27:30.437681 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:27:30.437681 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:27:30.437681 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:27:30.437681 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:27:30.437681 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:27:30.437681 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:27:30.437681 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:27:30.437681 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 17:27:30.437681 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Aug 24 17:27:30.437681 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 166/339] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Aug 24 17:27:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:27:30.443927 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbe9c9a-3098-4294-9ee3-9a9b83976f11 None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:30.563752 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9645b142-1819-4f7e-a7d5-90a801a9fc8e None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:30.579163 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbe9c9a-3098-4294-9ee3-9a9b83976f11 None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:30.615476 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:30.634335 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7488074-1f99-405b-85af-ef51994df471 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:30.634874 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7488074-1f99-405b-85af-ef51994df471 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:30.635693 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7488074-1f99-405b-85af-ef51994df471 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:30.635991 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7488074-1f99-405b-85af-ef51994df471 None admin] PATH_INFO: `/v3/roles` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:30.636719 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7488074-1f99-405b-85af-ef51994df471 None admin] RBAC: Adding query filter params (name=member) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:27:30.637015 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7488074-1f99-405b-85af-ef51994df471 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:30.638394 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7488074-1f99-405b-85af-ef51994df471 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:30.654596 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 174/340] 10.222.0.45 () {56 vars in 1101 bytes} [Wed Aug 24 17:27:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:27:30.660416 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7488074-1f99-405b-85af-ef51994df471 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:30.710980 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:30.746078 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9645b142-1819-4f7e-a7d5-90a801a9fc8e None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:30.810566 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0848ed5-8963-4351-aec2-3aa6c34682c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:30.811187 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0848ed5-8963-4351-aec2-3aa6c34682c1 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:30.811467 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0848ed5-8963-4351-aec2-3aa6c34682c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:30.811736 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0848ed5-8963-4351-aec2-3aa6c34682c1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:30.826072 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7488074-1f99-405b-85af-ef51994df471 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:30.827978 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0848ed5-8963-4351-aec2-3aa6c34682c1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:27:30.833386 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application [None req-c0848ed5-8963-4351-aec2-3aa6c34682c1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 17:27:30.833386 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:27:30.833386 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:27:30.833386 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:27:30.833386 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:27:30.833386 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:27:30.833386 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:27:30.833386 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:27:30.833386 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:27:30.833386 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:27:30.833386 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:27:30.833386 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:27:30.833386 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:27:30.833386 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:27:30.833386 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:27:30.833386 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:27:30.834435 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:27:30.834435 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:27:30.834435 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:27:30.834435 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:27:30.834435 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:27:30.834435 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:27:30.834435 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:27:30.834435 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:27:30.834435 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:27:30.834435 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:27:30.834435 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:27:30.834435 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application with Lock( Aug 24 17:27:30.834435 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:27:30.834435 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:27:30.834435 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:27:30.834435 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:27:30.835385 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:27:30.835385 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:27:30.835385 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:27:30.835385 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:27:30.835385 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:27:30.835385 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:27:30.835385 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:27:30.835385 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:27:30.835385 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:27:30.835385 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:27:30.835385 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 17:27:30.835385 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Aug 24 17:27:30.837313 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 167/341] 10.222.0.45 () {56 vars in 1114 bytes} [Wed Aug 24 17:27:30 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:27:30.846385 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0848ed5-8963-4351-aec2-3aa6c34682c1 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:30.916072 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b864fe-0d3d-4b7f-bece-28e2bf8efbc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:30.916758 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b864fe-0d3d-4b7f-bece-28e2bf8efbc2 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:30.917043 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b864fe-0d3d-4b7f-bece-28e2bf8efbc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:30.917316 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b864fe-0d3d-4b7f-bece-28e2bf8efbc2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:30.943682 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:30.957611 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0848ed5-8963-4351-aec2-3aa6c34682c1 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:30.992830 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5984fb-3532-49cd-8708-81615eedbb30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:30.993816 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5984fb-3532-49cd-8708-81615eedbb30 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:30.994284 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5984fb-3532-49cd-8708-81615eedbb30 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:30.995325 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5984fb-3532-49cd-8708-81615eedbb30 None admin] PATH_INFO: `/v3/roles` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:30.996179 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5984fb-3532-49cd-8708-81615eedbb30 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:27:30.996695 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5984fb-3532-49cd-8708-81615eedbb30 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:30.998268 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5984fb-3532-49cd-8708-81615eedbb30 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:31.006945 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 168/342] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Aug 24 17:27:30 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:27:31.008427 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00447cda-f316-40de-8542-a84a74eda0f8 None None] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:31.012991 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00447cda-f316-40de-8542-a84a74eda0f8 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:31.013489 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00447cda-f316-40de-8542-a84a74eda0f8 None None] PATH_INFO: `/` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:31.015307 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 169/343] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:27:31 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:27:31.017453 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00447cda-f316-40de-8542-a84a74eda0f8 None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:31.066655 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b864fe-0d3d-4b7f-bece-28e2bf8efbc2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:31.068141 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b864fe-0d3d-4b7f-bece-28e2bf8efbc2 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:31.131044 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00447cda-f316-40de-8542-a84a74eda0f8 None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:31.140902 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:31.170260 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9690bd-f90f-4723-a7ab-f4458c373334 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:31.170790 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9690bd-f90f-4723-a7ab-f4458c373334 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:31.171063 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9690bd-f90f-4723-a7ab-f4458c373334 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:31.171330 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9690bd-f90f-4723-a7ab-f4458c373334 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:31.192493 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:31.237260 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 175/344] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Aug 24 17:27:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 577 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 17:27:31.239551 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8f856a-e666-465d-9f17-afcc82d1b888 None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:31.240321 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8f856a-e666-465d-9f17-afcc82d1b888 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:31.244899 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8f856a-e666-465d-9f17-afcc82d1b888 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:31.262547 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9690bd-f90f-4723-a7ab-f4458c373334 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:31.264326 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9690bd-f90f-4723-a7ab-f4458c373334 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:31.265596 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-4a8f856a-e666-465d-9f17-afcc82d1b888 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:27:31.266772 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-4a8f856a-e666-465d-9f17-afcc82d1b888 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:27:31.345150 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 176/345] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:27:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:27:31.346546 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6793a2d-1814-4aa0-82b8-9564765b7ab0 None None] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:31.346732 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6793a2d-1814-4aa0-82b8-9564765b7ab0 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:31.347030 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6793a2d-1814-4aa0-82b8-9564765b7ab0 None None] PATH_INFO: `/` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:31.349573 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 177/346] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:27:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:27:31.350143 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6793a2d-1814-4aa0-82b8-9564765b7ab0 None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:31.399233 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 170/347] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Aug 24 17:27:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 17:27:31.401487 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6565c12-e46d-460e-b700-0b6bc906de17 None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:31.401983 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6565c12-e46d-460e-b700-0b6bc906de17 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:31.402554 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6565c12-e46d-460e-b700-0b6bc906de17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:31.422170 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-a6565c12-e46d-460e-b700-0b6bc906de17 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:27:31.423730 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-a6565c12-e46d-460e-b700-0b6bc906de17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:27:31.425519 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:31.469092 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6793a2d-1814-4aa0-82b8-9564765b7ab0 None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:31.480154 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:31.501007 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 171/348] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:27:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:27:31.502223 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6565c12-e46d-460e-b700-0b6bc906de17 None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:31.512351 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4448ba8-a648-461d-963f-f4bc4c7a090b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:31.513097 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4448ba8-a648-461d-963f-f4bc4c7a090b None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:31.513375 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4448ba8-a648-461d-963f-f4bc4c7a090b None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:31.513666 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4448ba8-a648-461d-963f-f4bc4c7a090b None admin] PATH_INFO: `/v3/projects/cc1182fc1781499884cebfaacd1ef5a0` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:31.515369 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4448ba8-a648-461d-963f-f4bc4c7a090b None admin] RBAC: Authorizing `identity:get_project(project_id=cc1182fc1781499884cebfaacd1ef5a0)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:31.516829 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4448ba8-a648-461d-963f-f4bc4c7a090b None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:31.518951 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 178/349] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Aug 24 17:27:31 2022] GET /identity/v3/projects/cc1182fc1781499884cebfaacd1ef5a0 => generated 298 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:27:31.527433 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4448ba8-a648-461d-963f-f4bc4c7a090b None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:31.619530 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6565c12-e46d-460e-b700-0b6bc906de17 None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:31.632432 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4448ba8-a648-461d-963f-f4bc4c7a090b None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:31.658034 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5f8c78-4a93-4253-a328-950c55739bf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:31.658912 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5f8c78-4a93-4253-a328-950c55739bf3 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:31.659370 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5f8c78-4a93-4253-a328-950c55739bf3 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:31.659789 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5f8c78-4a93-4253-a328-950c55739bf3 None admin] PATH_INFO: `/v3/projects/cc1182fc1781499884cebfaacd1ef5a0` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:31.661144 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5f8c78-4a93-4253-a328-950c55739bf3 None admin] RBAC: Authorizing `identity:get_project(project_id=cc1182fc1781499884cebfaacd1ef5a0)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:31.662762 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5f8c78-4a93-4253-a328-950c55739bf3 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:31.668312 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:31.672238 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 172/350] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Aug 24 17:27:31 2022] GET /identity/v3/projects/cc1182fc1781499884cebfaacd1ef5a0 => generated 298 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:27:31.673495 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5f8c78-4a93-4253-a328-950c55739bf3 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:31.675187 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80a87f6-a82a-4587-80b3-087d4d6786a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:31.675942 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80a87f6-a82a-4587-80b3-087d4d6786a4 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:31.676282 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80a87f6-a82a-4587-80b3-087d4d6786a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:31.676687 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80a87f6-a82a-4587-80b3-087d4d6786a4 None admin] PATH_INFO: `/v3/groups/cf46de7dc4bf44cb96323a2d60a4be51` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:31.689296 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80a87f6-a82a-4587-80b3-087d4d6786a4 None admin] RBAC: Authorizing `identity:get_group(group_id=cf46de7dc4bf44cb96323a2d60a4be51)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:31.690761 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80a87f6-a82a-4587-80b3-087d4d6786a4 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:31.694396 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 179/351] 10.222.0.45 () {56 vars in 1180 bytes} [Wed Aug 24 17:27:31 2022] GET /identity/v3/groups/cf46de7dc4bf44cb96323a2d60a4be51 => generated 226 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:27:31.702604 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80a87f6-a82a-4587-80b3-087d4d6786a4 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:31.781233 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:31.882629 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5f8c78-4a93-4253-a328-950c55739bf3 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:31.889914 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80a87f6-a82a-4587-80b3-087d4d6786a4 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:31.935528 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d8e60b-27d9-4525-90bd-270f2adda921 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:31.935710 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db5b2bc-0fd2-4505-9597-f0cc5aebcd97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:31.936066 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d8e60b-27d9-4525-90bd-270f2adda921 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:31.936173 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db5b2bc-0fd2-4505-9597-f0cc5aebcd97 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:31.936346 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d8e60b-27d9-4525-90bd-270f2adda921 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:31.936445 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db5b2bc-0fd2-4505-9597-f0cc5aebcd97 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:31.936642 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d8e60b-27d9-4525-90bd-270f2adda921 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:31.936755 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db5b2bc-0fd2-4505-9597-f0cc5aebcd97 None admin] PATH_INFO: `/v3/groups/cf46de7dc4bf44cb96323a2d60a4be51` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:31.937491 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d8e60b-27d9-4525-90bd-270f2adda921 None admin] RBAC: Adding query filter params (group.id=cf46de7dc4bf44cb96323a2d60a4be51, role.id=02476a5b82eb4b188c18a5d706a3ef68, scope.project.id=cc1182fc1781499884cebfaacd1ef5a0) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:27:31.937765 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db5b2bc-0fd2-4505-9597-f0cc5aebcd97 None admin] RBAC: Authorizing `identity:get_group(group_id=cf46de7dc4bf44cb96323a2d60a4be51)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:31.937877 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d8e60b-27d9-4525-90bd-270f2adda921 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:31.939205 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d8e60b-27d9-4525-90bd-270f2adda921 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:31.939319 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db5b2bc-0fd2-4505-9597-f0cc5aebcd97 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:31.941134 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 173/352] 10.222.0.45 () {56 vars in 1180 bytes} [Wed Aug 24 17:27:31 2022] GET /identity/v3/groups/cf46de7dc4bf44cb96323a2d60a4be51 => generated 226 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:27:31.941904 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db5b2bc-0fd2-4505-9597-f0cc5aebcd97 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:31.955004 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 180/353] 10.222.0.45 () {56 vars in 1376 bytes} [Wed Aug 24 17:27:31 2022] GET /identity/v3/role_assignments?group.id=cf46de7dc4bf44cb96323a2d60a4be51&scope.project.id=cc1182fc1781499884cebfaacd1ef5a0&role.id=02476a5b82eb4b188c18a5d706a3ef68 => generated 260 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:27:31.955931 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d8e60b-27d9-4525-90bd-270f2adda921 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:32.046129 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:32.055479 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db5b2bc-0fd2-4505-9597-f0cc5aebcd97 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:32.078099 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:32.094672 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed067eb3-2267-44ed-a147-d1e2ca2916e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:32.095314 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed067eb3-2267-44ed-a147-d1e2ca2916e4 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:32.095598 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed067eb3-2267-44ed-a147-d1e2ca2916e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:32.095870 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed067eb3-2267-44ed-a147-d1e2ca2916e4 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:32.108103 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed067eb3-2267-44ed-a147-d1e2ca2916e4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:27:32.111619 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d8e60b-27d9-4525-90bd-270f2adda921 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:32.113000 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application [None req-ed067eb3-2267-44ed-a147-d1e2ca2916e4 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 17:27:32.113000 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:27:32.113000 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:27:32.113000 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:27:32.113000 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:27:32.113000 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:27:32.113000 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:27:32.113000 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:27:32.113000 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:27:32.113000 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:27:32.113000 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:27:32.113000 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:27:32.113000 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:27:32.113000 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:27:32.113000 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:27:32.113000 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:27:32.113897 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:27:32.113897 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:27:32.113897 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:27:32.113897 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:27:32.113897 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:27:32.113897 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:27:32.113897 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:27:32.113897 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:27:32.113897 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:27:32.113897 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:27:32.113897 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:27:32.113897 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application with Lock( Aug 24 17:27:32.113897 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:27:32.113897 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:27:32.113897 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:27:32.113897 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:27:32.114678 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:27:32.114678 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:27:32.114678 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:27:32.114678 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:27:32.114678 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:27:32.114678 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:27:32.114678 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:27:32.114678 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:27:32.114678 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:27:32.114678 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:27:32.114678 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 17:27:32.114678 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Aug 24 17:27:32.114678 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 174/354] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Aug 24 17:27:31 2022] GET /identity/v3/roles/reader => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:27:32.115315 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed067eb3-2267-44ed-a147-d1e2ca2916e4 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:32.182079 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4797934a-9a14-4dd8-807d-91d911cec72c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:32.183017 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4797934a-9a14-4dd8-807d-91d911cec72c None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:32.183500 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4797934a-9a14-4dd8-807d-91d911cec72c None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:32.184008 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4797934a-9a14-4dd8-807d-91d911cec72c None admin] PATH_INFO: `/v3/roles/member` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:32.207805 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4797934a-9a14-4dd8-807d-91d911cec72c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:27:32.214149 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application [None req-4797934a-9a14-4dd8-807d-91d911cec72c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 17:27:32.214149 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:27:32.214149 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:27:32.214149 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:27:32.214149 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:27:32.214149 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:27:32.214149 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:27:32.214149 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:27:32.214149 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:27:32.214149 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:27:32.214149 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:27:32.214149 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:27:32.214149 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:27:32.214149 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:27:32.214149 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:27:32.214149 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:27:32.215216 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:27:32.215216 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:27:32.215216 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:27:32.215216 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:27:32.215216 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:27:32.215216 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:27:32.215216 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:27:32.215216 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:27:32.215216 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:27:32.215216 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:27:32.215216 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:27:32.215216 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application with Lock( Aug 24 17:27:32.215216 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:27:32.215216 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:27:32.215216 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:27:32.215216 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:27:32.216164 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:27:32.216164 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:27:32.216164 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:27:32.216164 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:27:32.216164 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:27:32.216164 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:27:32.216164 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:27:32.216164 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:27:32.216164 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:27:32.216164 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:27:32.216164 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 17:27:32.216164 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Aug 24 17:27:32.218512 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 181/355] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Aug 24 17:27:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:27:32.223830 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4797934a-9a14-4dd8-807d-91d911cec72c None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:32.227306 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed067eb3-2267-44ed-a147-d1e2ca2916e4 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:32.265493 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784515b1-b7fb-466e-8a5a-b8cd0bbab6e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:32.266077 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784515b1-b7fb-466e-8a5a-b8cd0bbab6e9 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:32.266353 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784515b1-b7fb-466e-8a5a-b8cd0bbab6e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:32.266622 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784515b1-b7fb-466e-8a5a-b8cd0bbab6e9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:32.267350 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784515b1-b7fb-466e-8a5a-b8cd0bbab6e9 None admin] RBAC: Adding query filter params (group.id=cf46de7dc4bf44cb96323a2d60a4be51, role.id=148f135e4a134697a032be04bc9faa4c, scope.project.id=cc1182fc1781499884cebfaacd1ef5a0) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:27:32.267659 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784515b1-b7fb-466e-8a5a-b8cd0bbab6e9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:32.270330 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784515b1-b7fb-466e-8a5a-b8cd0bbab6e9 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:32.278116 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 175/356] 10.222.0.45 () {56 vars in 1376 bytes} [Wed Aug 24 17:27:32 2022] GET /identity/v3/role_assignments?group.id=cf46de7dc4bf44cb96323a2d60a4be51&scope.project.id=cc1182fc1781499884cebfaacd1ef5a0&role.id=148f135e4a134697a032be04bc9faa4c => generated 260 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:27:32.278976 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784515b1-b7fb-466e-8a5a-b8cd0bbab6e9 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:32.323957 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:32.389915 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784515b1-b7fb-466e-8a5a-b8cd0bbab6e9 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:32.420444 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4797934a-9a14-4dd8-807d-91d911cec72c None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:32.429767 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463ad28b-eea4-40fa-b7c0-2f9d4400d30b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:32.430407 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463ad28b-eea4-40fa-b7c0-2f9d4400d30b None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:32.430688 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463ad28b-eea4-40fa-b7c0-2f9d4400d30b None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:32.430964 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463ad28b-eea4-40fa-b7c0-2f9d4400d30b None admin] PATH_INFO: `/v3/roles` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:32.431659 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463ad28b-eea4-40fa-b7c0-2f9d4400d30b None admin] RBAC: Adding query filter params (name=member) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:27:32.432024 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463ad28b-eea4-40fa-b7c0-2f9d4400d30b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:32.433986 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463ad28b-eea4-40fa-b7c0-2f9d4400d30b None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:32.443110 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 176/357] 10.222.0.45 () {56 vars in 1101 bytes} [Wed Aug 24 17:27:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:27:32.444366 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88267229-cbb5-4e1f-8edb-bed8ae243016 None None] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:32.448642 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88267229-cbb5-4e1f-8edb-bed8ae243016 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:32.448642 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88267229-cbb5-4e1f-8edb-bed8ae243016 None None] PATH_INFO: `/` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:32.448642 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 177/358] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:27:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:27:32.450587 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88267229-cbb5-4e1f-8edb-bed8ae243016 None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:32.472044 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:32.492038 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a3dfc1-4b9c-4bb7-9432-faa1a0a4b74c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:32.492989 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a3dfc1-4b9c-4bb7-9432-faa1a0a4b74c None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:32.493415 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a3dfc1-4b9c-4bb7-9432-faa1a0a4b74c None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:32.493832 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a3dfc1-4b9c-4bb7-9432-faa1a0a4b74c None admin] PATH_INFO: `/v3/roles` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:32.497709 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a3dfc1-4b9c-4bb7-9432-faa1a0a4b74c None admin] RBAC: Adding query filter params (name=reader) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:27:32.498357 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a3dfc1-4b9c-4bb7-9432-faa1a0a4b74c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:32.500169 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a3dfc1-4b9c-4bb7-9432-faa1a0a4b74c None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:32.510420 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 182/359] 10.222.0.45 () {56 vars in 1101 bytes} [Wed Aug 24 17:27:32 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:27:32.512210 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24ecf12-405e-4148-ac4a-b019887d2ac6 None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:32.512673 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24ecf12-405e-4148-ac4a-b019887d2ac6 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:32.513120 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24ecf12-405e-4148-ac4a-b019887d2ac6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:32.535485 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-f24ecf12-405e-4148-ac4a-b019887d2ac6 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:27:32.538954 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-f24ecf12-405e-4148-ac4a-b019887d2ac6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:27:32.634820 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 183/360] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:27:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:27:32.635495 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24ecf12-405e-4148-ac4a-b019887d2ac6 None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:32.697985 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88267229-cbb5-4e1f-8edb-bed8ae243016 None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:32.700100 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:32.758846 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ca4fe0-3166-4720-b31d-e2790f29067e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:32.761257 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ca4fe0-3166-4720-b31d-e2790f29067e None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:32.761705 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ca4fe0-3166-4720-b31d-e2790f29067e None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:32.762137 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ca4fe0-3166-4720-b31d-e2790f29067e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:32.826467 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24ecf12-405e-4148-ac4a-b019887d2ac6 None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:32.859887 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ca4fe0-3166-4720-b31d-e2790f29067e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:32.861913 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ca4fe0-3166-4720-b31d-e2790f29067e None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:32.863496 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:32.935834 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b638252-fa7c-4fda-8ec9-1558befa37bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:32.936493 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b638252-fa7c-4fda-8ec9-1558befa37bb None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:32.936798 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b638252-fa7c-4fda-8ec9-1558befa37bb None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:32.937069 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b638252-fa7c-4fda-8ec9-1558befa37bb None admin] PATH_INFO: `/v3/users/15ce3a91b6d84377941e9c83e617bd46` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:32.938223 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b638252-fa7c-4fda-8ec9-1558befa37bb None admin] RBAC: Authorizing `identity:get_user(user_id=15ce3a91b6d84377941e9c83e617bd46)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:32.939849 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b638252-fa7c-4fda-8ec9-1558befa37bb None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:32.941545 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 184/361] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Aug 24 17:27:32 2022] GET /identity/v3/users/15ce3a91b6d84377941e9c83e617bd46 => generated 293 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:27:32.947313 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b638252-fa7c-4fda-8ec9-1558befa37bb None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:32.982003 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 178/362] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Aug 24 17:27:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 531 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 17:27:32.988536 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ca4fe0-3166-4720-b31d-e2790f29067e None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:32.991383 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:33.136167 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ca4fe0-3166-4720-b31d-e2790f29067e None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:33.185034 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2641c5-6eb1-478c-aa69-710fb8ac5364 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:33.185945 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2641c5-6eb1-478c-aa69-710fb8ac5364 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:33.186368 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2641c5-6eb1-478c-aa69-710fb8ac5364 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:33.186839 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2641c5-6eb1-478c-aa69-710fb8ac5364 None admin] PATH_INFO: `/v3/projects/3a29643b449b4fc1aa71802d8db7c0d2` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:33.188174 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2641c5-6eb1-478c-aa69-710fb8ac5364 None admin] RBAC: Authorizing `identity:get_project(project_id=3a29643b449b4fc1aa71802d8db7c0d2)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:33.190376 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2641c5-6eb1-478c-aa69-710fb8ac5364 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:33.192697 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 179/363] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Aug 24 17:27:32 2022] GET /identity/v3/projects/3a29643b449b4fc1aa71802d8db7c0d2 => generated 302 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:27:33.198261 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2641c5-6eb1-478c-aa69-710fb8ac5364 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:33.221700 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b638252-fa7c-4fda-8ec9-1558befa37bb None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:33.282421 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:33.290242 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f7d1b0-19e7-454d-8526-fce8986cf5a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:33.290873 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f7d1b0-19e7-454d-8526-fce8986cf5a5 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:33.291156 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f7d1b0-19e7-454d-8526-fce8986cf5a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:33.291476 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f7d1b0-19e7-454d-8526-fce8986cf5a5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:33.292215 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f7d1b0-19e7-454d-8526-fce8986cf5a5 None admin] RBAC: Adding query filter params (role.id=645db6d227604357b1949889630755ff, user.id=15ce3a91b6d84377941e9c83e617bd46, scope.system=all) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:27:33.292529 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f7d1b0-19e7-454d-8526-fce8986cf5a5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:33.294280 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f7d1b0-19e7-454d-8526-fce8986cf5a5 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:33.308586 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 185/364] 10.222.0.45 () {56 vars in 1308 bytes} [Wed Aug 24 17:27:32 2022] GET /identity/v3/role_assignments?user.id=15ce3a91b6d84377941e9c83e617bd46&scope.system=all&role.id=645db6d227604357b1949889630755ff => generated 510 bytes in 361 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:27:33.313019 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f7d1b0-19e7-454d-8526-fce8986cf5a5 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:33.317300 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2641c5-6eb1-478c-aa69-710fb8ac5364 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:33.366571 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f0f43c-9f02-4124-9976-a4bc1cdbd9bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:33.367234 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f0f43c-9f02-4124-9976-a4bc1cdbd9bc None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:33.367518 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f0f43c-9f02-4124-9976-a4bc1cdbd9bc None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:33.367794 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f0f43c-9f02-4124-9976-a4bc1cdbd9bc None admin] PATH_INFO: `/v3/groups/cf46de7dc4bf44cb96323a2d60a4be51` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:33.370998 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f0f43c-9f02-4124-9976-a4bc1cdbd9bc None admin] RBAC: Authorizing `identity:get_group(group_id=cf46de7dc4bf44cb96323a2d60a4be51)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:33.372475 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f0f43c-9f02-4124-9976-a4bc1cdbd9bc None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:33.379030 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 180/365] 10.222.0.45 () {56 vars in 1180 bytes} [Wed Aug 24 17:27:33 2022] GET /identity/v3/groups/cf46de7dc4bf44cb96323a2d60a4be51 => generated 226 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:27:33.380320 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82879186-7a6a-4abb-beee-0bc5908d4467 None None] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:33.380623 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82879186-7a6a-4abb-beee-0bc5908d4467 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:33.380901 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82879186-7a6a-4abb-beee-0bc5908d4467 None None] PATH_INFO: `/` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:33.382244 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 181/366] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:27:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:27:33.387705 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82879186-7a6a-4abb-beee-0bc5908d4467 None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:33.412767 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:33.508512 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f7d1b0-19e7-454d-8526-fce8986cf5a5 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:33.564511 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66435cf6-a5f3-48d8-ab7c-174365f26e61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:33.565201 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66435cf6-a5f3-48d8-ab7c-174365f26e61 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:33.565484 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66435cf6-a5f3-48d8-ab7c-174365f26e61 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:33.565764 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66435cf6-a5f3-48d8-ab7c-174365f26e61 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:33.580979 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66435cf6-a5f3-48d8-ab7c-174365f26e61 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:27:33.588976 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application [None req-66435cf6-a5f3-48d8-ab7c-174365f26e61 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 17:27:33.588976 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:27:33.588976 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:27:33.588976 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:27:33.588976 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:27:33.588976 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:27:33.588976 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:27:33.588976 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:27:33.588976 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:27:33.588976 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:27:33.588976 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:27:33.588976 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:27:33.588976 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:27:33.588976 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:27:33.588976 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:27:33.588976 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:27:33.589877 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:27:33.589877 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:27:33.589877 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:27:33.589877 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:27:33.589877 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:27:33.589877 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:27:33.589877 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:27:33.589877 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:27:33.589877 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:27:33.589877 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:27:33.589877 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:27:33.589877 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application with Lock( Aug 24 17:27:33.589877 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:27:33.589877 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:27:33.589877 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:27:33.589877 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:27:33.590671 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:27:33.590671 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:27:33.590671 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:27:33.590671 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:27:33.590671 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:27:33.590671 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:27:33.590671 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:27:33.590671 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:27:33.590671 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:27:33.590671 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:27:33.590671 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 17:27:33.590671 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Aug 24 17:27:33.590671 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 186/367] 10.222.0.45 () {56 vars in 1114 bytes} [Wed Aug 24 17:27:33 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:27:33.591811 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb73540-19bd-4989-921c-ac5dec7b1a3d None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:33.591912 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb73540-19bd-4989-921c-ac5dec7b1a3d None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:33.595820 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb73540-19bd-4989-921c-ac5dec7b1a3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:33.603264 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82879186-7a6a-4abb-beee-0bc5908d4467 None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:33.613820 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-ecb73540-19bd-4989-921c-ac5dec7b1a3d None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:27:33.615032 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-ecb73540-19bd-4989-921c-ac5dec7b1a3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:27:33.648481 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f63c90-e1aa-4053-9fd2-192e84f3c152 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:33.649180 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f63c90-e1aa-4053-9fd2-192e84f3c152 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:33.649463 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f63c90-e1aa-4053-9fd2-192e84f3c152 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:33.649741 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f63c90-e1aa-4053-9fd2-192e84f3c152 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:33.650475 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f63c90-e1aa-4053-9fd2-192e84f3c152 None admin] RBAC: Adding query filter params (group.id=cf46de7dc4bf44cb96323a2d60a4be51, role.id=02476a5b82eb4b188c18a5d706a3ef68, scope.project.id=3a29643b449b4fc1aa71802d8db7c0d2) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:27:33.650889 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f63c90-e1aa-4053-9fd2-192e84f3c152 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:33.652400 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f63c90-e1aa-4053-9fd2-192e84f3c152 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:33.660714 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 182/368] 10.222.0.45 () {56 vars in 1376 bytes} [Wed Aug 24 17:27:33 2022] GET /identity/v3/role_assignments?group.id=cf46de7dc4bf44cb96323a2d60a4be51&scope.project.id=3a29643b449b4fc1aa71802d8db7c0d2&role.id=02476a5b82eb4b188c18a5d706a3ef68 => generated 260 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:27:33.661558 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f63c90-e1aa-4053-9fd2-192e84f3c152 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:33.669107 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:33.764585 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 187/369] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:27:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:27:33.793349 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f63c90-e1aa-4053-9fd2-192e84f3c152 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:33.846273 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d613c116-c9d4-4252-9d42-ad91d3903023 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:33.846901 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d613c116-c9d4-4252-9d42-ad91d3903023 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:33.847182 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d613c116-c9d4-4252-9d42-ad91d3903023 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:33.847448 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d613c116-c9d4-4252-9d42-ad91d3903023 None admin] PATH_INFO: `/v3/roles` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:33.848132 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d613c116-c9d4-4252-9d42-ad91d3903023 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:27:33.848431 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d613c116-c9d4-4252-9d42-ad91d3903023 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:33.849845 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d613c116-c9d4-4252-9d42-ad91d3903023 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:33.860083 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 183/370] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Aug 24 17:27:33 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:27:33.866326 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb73540-19bd-4989-921c-ac5dec7b1a3d None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:33.911974 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:33.998032 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb73540-19bd-4989-921c-ac5dec7b1a3d None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:34.041818 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59868fd2-9b2b-443e-9250-4b52b34199a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:34.042449 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59868fd2-9b2b-443e-9250-4b52b34199a5 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:34.042733 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59868fd2-9b2b-443e-9250-4b52b34199a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:34.043010 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59868fd2-9b2b-443e-9250-4b52b34199a5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:34.128596 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59868fd2-9b2b-443e-9250-4b52b34199a5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:34.130054 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59868fd2-9b2b-443e-9250-4b52b34199a5 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:34.163165 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:34.245408 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 188/371] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Aug 24 17:27:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 17:27:34.247976 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d613c116-c9d4-4252-9d42-ad91d3903023 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:34.262779 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59868fd2-9b2b-443e-9250-4b52b34199a5 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:34.334923 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:34.382291 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d613c116-c9d4-4252-9d42-ad91d3903023 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:34.395291 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59868fd2-9b2b-443e-9250-4b52b34199a5 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:34.441098 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:34.449075 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b32787-38dc-4fc6-b2dc-3feb15082ae9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:34.449724 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b32787-38dc-4fc6-b2dc-3feb15082ae9 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:34.450007 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b32787-38dc-4fc6-b2dc-3feb15082ae9 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:34.450279 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b32787-38dc-4fc6-b2dc-3feb15082ae9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:34.457738 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8d34ef-dfc4-4ccd-a9eb-1189e68c337d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:34.458380 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8d34ef-dfc4-4ccd-a9eb-1189e68c337d None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:34.458661 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8d34ef-dfc4-4ccd-a9eb-1189e68c337d None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:34.458934 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8d34ef-dfc4-4ccd-a9eb-1189e68c337d None admin] PATH_INFO: `/v3/projects/3a29643b449b4fc1aa71802d8db7c0d2` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:34.460232 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8d34ef-dfc4-4ccd-a9eb-1189e68c337d None admin] RBAC: Authorizing `identity:get_project(project_id=3a29643b449b4fc1aa71802d8db7c0d2)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:34.463895 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8d34ef-dfc4-4ccd-a9eb-1189e68c337d None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:34.467031 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 189/372] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Aug 24 17:27:34 2022] GET /identity/v3/projects/3a29643b449b4fc1aa71802d8db7c0d2 => generated 302 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:27:34.471711 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8d34ef-dfc4-4ccd-a9eb-1189e68c337d None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:34.475316 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b32787-38dc-4fc6-b2dc-3feb15082ae9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:27:34.481294 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application [None req-64b32787-38dc-4fc6-b2dc-3feb15082ae9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:27:34.481294 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:27:34.481294 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:27:34.481294 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:27:34.481294 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:27:34.481294 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:27:34.481294 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:27:34.481294 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:27:34.481294 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:27:34.481294 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:27:34.481294 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:27:34.481294 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:27:34.481294 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:27:34.481294 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:27:34.481294 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:27:34.481294 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:27:34.483264 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:27:34.483264 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:27:34.483264 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:27:34.483264 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:27:34.483264 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:27:34.483264 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:27:34.483264 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:27:34.483264 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:27:34.483264 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:27:34.483264 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:27:34.483264 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:27:34.483264 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application with Lock( Aug 24 17:27:34.483264 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:27:34.483264 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:27:34.483264 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:27:34.483264 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:27:34.484299 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:27:34.484299 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:27:34.484299 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:27:34.484299 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:27:34.484299 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:27:34.484299 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:27:34.484299 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:27:34.484299 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:27:34.484299 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:27:34.484299 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:27:34.484299 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:27:34.484299 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Aug 24 17:27:34.484299 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 184/373] 10.222.0.45 () {56 vars in 1096 bytes} [Wed Aug 24 17:27:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:27:34.497234 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b32787-38dc-4fc6-b2dc-3feb15082ae9 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:34.595752 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8d34ef-dfc4-4ccd-a9eb-1189e68c337d None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:34.632971 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae2fd20-3fb7-40f7-b8db-c23ed16a25c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:34.633457 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae2fd20-3fb7-40f7-b8db-c23ed16a25c4 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:34.633737 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae2fd20-3fb7-40f7-b8db-c23ed16a25c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:34.634001 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae2fd20-3fb7-40f7-b8db-c23ed16a25c4 None admin] PATH_INFO: `/v3/groups/cf46de7dc4bf44cb96323a2d60a4be51` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:34.635169 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae2fd20-3fb7-40f7-b8db-c23ed16a25c4 None admin] RBAC: Authorizing `identity:get_group(group_id=cf46de7dc4bf44cb96323a2d60a4be51)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:34.636641 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae2fd20-3fb7-40f7-b8db-c23ed16a25c4 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:34.638659 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 190/374] 10.222.0.45 () {56 vars in 1180 bytes} [Wed Aug 24 17:27:34 2022] GET /identity/v3/groups/cf46de7dc4bf44cb96323a2d60a4be51 => generated 226 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:27:34.639591 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef75fec-09e9-488c-95a5-679569a986fa None None] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:34.639870 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef75fec-09e9-488c-95a5-679569a986fa None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:34.640142 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef75fec-09e9-488c-95a5-679569a986fa None None] PATH_INFO: `/` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:34.645519 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 191/375] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:27:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:27:34.645519 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef75fec-09e9-488c-95a5-679569a986fa None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:34.652492 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b32787-38dc-4fc6-b2dc-3feb15082ae9 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:34.677215 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:34.689941 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700c7fca-ffc6-4510-a82b-b3e599f52fcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:34.690750 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700c7fca-ffc6-4510-a82b-b3e599f52fcb None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:34.691177 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700c7fca-ffc6-4510-a82b-b3e599f52fcb None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:34.691492 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700c7fca-ffc6-4510-a82b-b3e599f52fcb None admin] PATH_INFO: `/v3/roles` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:34.692264 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700c7fca-ffc6-4510-a82b-b3e599f52fcb None admin] RBAC: Adding query filter params (name=admin) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:27:34.692736 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700c7fca-ffc6-4510-a82b-b3e599f52fcb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:34.694275 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700c7fca-ffc6-4510-a82b-b3e599f52fcb None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:34.702995 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 185/376] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Aug 24 17:27:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:27:34.708622 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1396a107-7e35-4e62-8740-7fc74fb011aa None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:34.708622 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1396a107-7e35-4e62-8740-7fc74fb011aa None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:34.708622 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1396a107-7e35-4e62-8740-7fc74fb011aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:34.726260 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-1396a107-7e35-4e62-8740-7fc74fb011aa None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:27:34.727419 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-1396a107-7e35-4e62-8740-7fc74fb011aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:27:34.753020 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:34.761290 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef75fec-09e9-488c-95a5-679569a986fa None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:34.804499 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f57302-d15e-427f-9b08-d3859e0873ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:34.804499 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f57302-d15e-427f-9b08-d3859e0873ba None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:34.804499 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f57302-d15e-427f-9b08-d3859e0873ba None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:34.804499 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f57302-d15e-427f-9b08-d3859e0873ba None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:34.804857 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f57302-d15e-427f-9b08-d3859e0873ba None admin] RBAC: Adding query filter params (group.id=cf46de7dc4bf44cb96323a2d60a4be51, role.id=148f135e4a134697a032be04bc9faa4c, scope.project.id=3a29643b449b4fc1aa71802d8db7c0d2) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:27:34.804857 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f57302-d15e-427f-9b08-d3859e0873ba None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:34.804975 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 186/377] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:27:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:27:34.804975 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1396a107-7e35-4e62-8740-7fc74fb011aa None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:34.805106 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f57302-d15e-427f-9b08-d3859e0873ba None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:34.810580 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 192/378] 10.222.0.45 () {56 vars in 1376 bytes} [Wed Aug 24 17:27:34 2022] GET /identity/v3/role_assignments?group.id=cf46de7dc4bf44cb96323a2d60a4be51&scope.project.id=3a29643b449b4fc1aa71802d8db7c0d2&role.id=148f135e4a134697a032be04bc9faa4c => generated 260 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:27:34.812432 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e382518d-2790-4479-9c28-987df31b977a None None] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:34.812749 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e382518d-2790-4479-9c28-987df31b977a None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:34.813026 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e382518d-2790-4479-9c28-987df31b977a None None] PATH_INFO: `/` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:34.814498 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 193/379] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:27:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:27:34.822099 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b3b36a-eb20-4c1d-8f1a-8f162f5f6352 None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:34.822444 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b3b36a-eb20-4c1d-8f1a-8f162f5f6352 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:34.822690 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b3b36a-eb20-4c1d-8f1a-8f162f5f6352 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:34.842342 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-98b3b36a-eb20-4c1d-8f1a-8f162f5f6352 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:27:34.843694 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-98b3b36a-eb20-4c1d-8f1a-8f162f5f6352 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:27:34.970622 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 194/380] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:27:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:27:35.013359 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1396a107-7e35-4e62-8740-7fc74fb011aa None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:35.032179 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:35.052199 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac8d892-18e5-4848-958e-4e737d2dc0e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:35.053082 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac8d892-18e5-4848-958e-4e737d2dc0e3 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:35.053504 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac8d892-18e5-4848-958e-4e737d2dc0e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:35.053919 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac8d892-18e5-4848-958e-4e737d2dc0e3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:35.109244 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b3b36a-eb20-4c1d-8f1a-8f162f5f6352 None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:35.159916 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac8d892-18e5-4848-958e-4e737d2dc0e3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:35.161416 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac8d892-18e5-4848-958e-4e737d2dc0e3 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:35.222137 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:35.297219 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b3b36a-eb20-4c1d-8f1a-8f162f5f6352 None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:35.346628 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 187/381] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Aug 24 17:27:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 543 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 17:27:35.348100 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac8d892-18e5-4848-958e-4e737d2dc0e3 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:35.350338 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1f14d7-c746-4eee-9eaa-db7c921b22fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:35.351763 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1f14d7-c746-4eee-9eaa-db7c921b22fe None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:35.352117 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1f14d7-c746-4eee-9eaa-db7c921b22fe None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:35.352457 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1f14d7-c746-4eee-9eaa-db7c921b22fe None admin] PATH_INFO: `/v3/roles/member` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:35.368858 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:35.373470 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1f14d7-c746-4eee-9eaa-db7c921b22fe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:27:35.387012 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application [None req-9b1f14d7-c746-4eee-9eaa-db7c921b22fe None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 17:27:35.387012 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:27:35.387012 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:27:35.387012 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:27:35.387012 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:27:35.387012 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:27:35.387012 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:27:35.387012 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:27:35.387012 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:27:35.387012 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:27:35.387012 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:27:35.387012 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:27:35.387012 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:27:35.387012 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:27:35.387012 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:27:35.387012 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:27:35.387922 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:27:35.387922 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:27:35.387922 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:27:35.387922 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:27:35.387922 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:27:35.387922 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:27:35.387922 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:27:35.387922 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:27:35.387922 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:27:35.387922 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:27:35.387922 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:27:35.387922 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application with Lock( Aug 24 17:27:35.387922 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:27:35.387922 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:27:35.387922 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:27:35.387922 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:27:35.389138 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:27:35.389138 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:27:35.389138 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:27:35.389138 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:27:35.389138 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:27:35.389138 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:27:35.389138 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:27:35.389138 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:27:35.389138 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:27:35.389138 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:27:35.389138 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 17:27:35.389138 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Aug 24 17:27:35.389138 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 195/382] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Aug 24 17:27:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:27:35.389138 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1f14d7-c746-4eee-9eaa-db7c921b22fe None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:35.475316 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac8d892-18e5-4848-958e-4e737d2dc0e3 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:35.499358 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1f14d7-c746-4eee-9eaa-db7c921b22fe None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:35.519660 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:35.522811 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb76e735-d27b-4d33-904e-c46a64a9a89b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:35.525495 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb76e735-d27b-4d33-904e-c46a64a9a89b None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:35.526726 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb76e735-d27b-4d33-904e-c46a64a9a89b None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:35.527980 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb76e735-d27b-4d33-904e-c46a64a9a89b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:35.547055 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a395508f-2a35-4f6b-a5bb-be7fc7354912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:35.547717 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb76e735-d27b-4d33-904e-c46a64a9a89b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:27:35.548299 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a395508f-2a35-4f6b-a5bb-be7fc7354912 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:35.548777 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a395508f-2a35-4f6b-a5bb-be7fc7354912 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:35.549209 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a395508f-2a35-4f6b-a5bb-be7fc7354912 None admin] PATH_INFO: `/v3/projects/1e847003e2ef47df81d36ac4caad47f0` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:35.550542 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a395508f-2a35-4f6b-a5bb-be7fc7354912 None admin] RBAC: Authorizing `identity:get_project(project_id=1e847003e2ef47df81d36ac4caad47f0)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:35.552143 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a395508f-2a35-4f6b-a5bb-be7fc7354912 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:35.553375 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application [None req-fb76e735-d27b-4d33-904e-c46a64a9a89b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 17:27:35.553375 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:27:35.553375 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:27:35.553375 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:27:35.553375 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:27:35.553375 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:27:35.553375 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:27:35.553375 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:27:35.553375 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:27:35.553375 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:27:35.553375 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:27:35.553375 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:27:35.553375 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:27:35.553375 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:27:35.553375 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:27:35.553375 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:27:35.554329 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:27:35.554329 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:27:35.554329 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:27:35.554329 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:27:35.554329 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:27:35.554329 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:27:35.554329 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:27:35.554329 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:27:35.554329 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:27:35.554329 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:27:35.554329 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:27:35.554329 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application with Lock( Aug 24 17:27:35.554329 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:27:35.554329 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:27:35.554329 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:27:35.554329 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:27:35.555352 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:27:35.555352 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:27:35.555352 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:27:35.555352 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:27:35.555352 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:27:35.555352 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:27:35.555352 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:27:35.555352 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:27:35.555352 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:27:35.555352 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:27:35.555352 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 17:27:35.555352 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Aug 24 17:27:35.555352 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 188/383] 10.222.0.45 () {56 vars in 1114 bytes} [Wed Aug 24 17:27:35 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:27:35.556432 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb76e735-d27b-4d33-904e-c46a64a9a89b None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:35.557271 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 196/384] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Aug 24 17:27:35 2022] GET /identity/v3/projects/1e847003e2ef47df81d36ac4caad47f0 => generated 344 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:27:35.562385 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a395508f-2a35-4f6b-a5bb-be7fc7354912 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:35.652242 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:35.685378 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb76e735-d27b-4d33-904e-c46a64a9a89b None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:35.692304 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a395508f-2a35-4f6b-a5bb-be7fc7354912 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:35.731362 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063c4133-5db6-4e4c-af5c-1ac0a2af0cf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:35.732285 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063c4133-5db6-4e4c-af5c-1ac0a2af0cf7 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:35.732775 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063c4133-5db6-4e4c-af5c-1ac0a2af0cf7 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:35.733243 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063c4133-5db6-4e4c-af5c-1ac0a2af0cf7 None admin] PATH_INFO: `/v3/roles` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:35.734086 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063c4133-5db6-4e4c-af5c-1ac0a2af0cf7 None admin] RBAC: Adding query filter params (name=member) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:27:35.734570 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063c4133-5db6-4e4c-af5c-1ac0a2af0cf7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:35.734986 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bed5bf2-3720-462d-a242-92498804adec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:35.735675 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bed5bf2-3720-462d-a242-92498804adec None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:35.736027 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bed5bf2-3720-462d-a242-92498804adec None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:35.736369 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bed5bf2-3720-462d-a242-92498804adec None admin] PATH_INFO: `/v3/roles` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:35.737572 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bed5bf2-3720-462d-a242-92498804adec None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:27:35.737572 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bed5bf2-3720-462d-a242-92498804adec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:35.738187 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063c4133-5db6-4e4c-af5c-1ac0a2af0cf7 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:35.738960 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bed5bf2-3720-462d-a242-92498804adec None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:35.752585 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 197/386] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Aug 24 17:27:35 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:27:35.752585 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bed5bf2-3720-462d-a242-92498804adec None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:35.752976 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 189/386] 10.222.0.45 () {56 vars in 1101 bytes} [Wed Aug 24 17:27:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:27:35.755535 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063c4133-5db6-4e4c-af5c-1ac0a2af0cf7 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:35.832435 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:35.905208 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063c4133-5db6-4e4c-af5c-1ac0a2af0cf7 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:35.912470 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bed5bf2-3720-462d-a242-92498804adec None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:35.953899 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5232a02-5291-41d3-bd10-e995f1314917 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:35.954533 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5232a02-5291-41d3-bd10-e995f1314917 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:35.954814 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5232a02-5291-41d3-bd10-e995f1314917 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:35.955089 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5232a02-5291-41d3-bd10-e995f1314917 None admin] PATH_INFO: `/v3/groups/cf46de7dc4bf44cb96323a2d60a4be51` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:35.956188 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5232a02-5291-41d3-bd10-e995f1314917 None admin] RBAC: Authorizing `identity:get_group(group_id=cf46de7dc4bf44cb96323a2d60a4be51)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:35.957763 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5232a02-5291-41d3-bd10-e995f1314917 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:35.959522 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 190/387] 10.222.0.45 () {56 vars in 1180 bytes} [Wed Aug 24 17:27:35 2022] GET /identity/v3/groups/cf46de7dc4bf44cb96323a2d60a4be51 => generated 226 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:27:35.960503 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffc9f1f-0687-49a2-b189-414a13818356 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:35.961415 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffc9f1f-0687-49a2-b189-414a13818356 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:35.961893 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffc9f1f-0687-49a2-b189-414a13818356 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:35.962388 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffc9f1f-0687-49a2-b189-414a13818356 None admin] PATH_INFO: `/v3/groups/5877fa6c407248d4b6f9fc1a15676f44` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:35.965335 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5232a02-5291-41d3-bd10-e995f1314917 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:35.968513 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffc9f1f-0687-49a2-b189-414a13818356 None admin] RBAC: Authorizing `identity:get_group(group_id=5877fa6c407248d4b6f9fc1a15676f44)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:35.970255 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffc9f1f-0687-49a2-b189-414a13818356 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:35.972383 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 198/388] 10.222.0.45 () {56 vars in 1180 bytes} [Wed Aug 24 17:27:35 2022] GET /identity/v3/groups/5877fa6c407248d4b6f9fc1a15676f44 => generated 229 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:27:35.973400 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffc9f1f-0687-49a2-b189-414a13818356 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:35.974996 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:36.074838 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5232a02-5291-41d3-bd10-e995f1314917 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:36.084501 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffc9f1f-0687-49a2-b189-414a13818356 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:36.098309 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:36.111349 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e968e2-ebab-4d87-8520-1adafcde9ffc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:36.112014 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e968e2-ebab-4d87-8520-1adafcde9ffc None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:36.112298 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e968e2-ebab-4d87-8520-1adafcde9ffc None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:36.112750 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e968e2-ebab-4d87-8520-1adafcde9ffc None admin] PATH_INFO: `/v3/groups/cf46de7dc4bf44cb96323a2d60a4be51` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:36.113984 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e968e2-ebab-4d87-8520-1adafcde9ffc None admin] RBAC: Authorizing `identity:get_group(group_id=cf46de7dc4bf44cb96323a2d60a4be51)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:36.115465 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e968e2-ebab-4d87-8520-1adafcde9ffc None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:36.122500 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 191/389] 10.222.0.45 () {56 vars in 1180 bytes} [Wed Aug 24 17:27:35 2022] GET /identity/v3/groups/cf46de7dc4bf44cb96323a2d60a4be51 => generated 226 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:27:36.123720 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e968e2-ebab-4d87-8520-1adafcde9ffc None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:36.127197 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b0ef9a-36ad-48c8-b6a7-a479475771de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:36.128595 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b0ef9a-36ad-48c8-b6a7-a479475771de None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:36.129091 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b0ef9a-36ad-48c8-b6a7-a479475771de None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:36.129561 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b0ef9a-36ad-48c8-b6a7-a479475771de None admin] PATH_INFO: `/v3/projects/cc1182fc1781499884cebfaacd1ef5a0` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:36.131033 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b0ef9a-36ad-48c8-b6a7-a479475771de None admin] RBAC: Authorizing `identity:get_project(project_id=cc1182fc1781499884cebfaacd1ef5a0)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:36.132719 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b0ef9a-36ad-48c8-b6a7-a479475771de None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:36.134597 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 199/390] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Aug 24 17:27:35 2022] GET /identity/v3/projects/cc1182fc1781499884cebfaacd1ef5a0 => generated 298 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:27:36.135638 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b0ef9a-36ad-48c8-b6a7-a479475771de None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:36.216622 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:36.252588 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e968e2-ebab-4d87-8520-1adafcde9ffc None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:36.263587 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b0ef9a-36ad-48c8-b6a7-a479475771de None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:36.291841 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99af8d4-b10d-427d-9dea-4ef5fd11fb28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:36.292460 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99af8d4-b10d-427d-9dea-4ef5fd11fb28 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:36.292777 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99af8d4-b10d-427d-9dea-4ef5fd11fb28 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:36.293133 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99af8d4-b10d-427d-9dea-4ef5fd11fb28 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:36.293778 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99af8d4-b10d-427d-9dea-4ef5fd11fb28 None admin] RBAC: Adding query filter params (group.id=5877fa6c407248d4b6f9fc1a15676f44, role.id=e411ad9eb96e4e8d85b59b5b700ada7b, scope.project.id=1e847003e2ef47df81d36ac4caad47f0) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:27:36.294089 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99af8d4-b10d-427d-9dea-4ef5fd11fb28 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:36.295524 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99af8d4-b10d-427d-9dea-4ef5fd11fb28 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:36.304346 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 192/391] 10.222.0.45 () {56 vars in 1376 bytes} [Wed Aug 24 17:27:36 2022] GET /identity/v3/role_assignments?group.id=5877fa6c407248d4b6f9fc1a15676f44&scope.project.id=1e847003e2ef47df81d36ac4caad47f0&role.id=e411ad9eb96e4e8d85b59b5b700ada7b => generated 260 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:27:36.305744 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99af8d4-b10d-427d-9dea-4ef5fd11fb28 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:36.306377 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35541d46-6f42-4a6a-86a7-f371a0de385e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:36.307114 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35541d46-6f42-4a6a-86a7-f371a0de385e None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:36.307951 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35541d46-6f42-4a6a-86a7-f371a0de385e None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:36.308305 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35541d46-6f42-4a6a-86a7-f371a0de385e None admin] PATH_INFO: `/v3/projects/cc1182fc1781499884cebfaacd1ef5a0` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:36.309596 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35541d46-6f42-4a6a-86a7-f371a0de385e None admin] RBAC: Authorizing `identity:get_project(project_id=cc1182fc1781499884cebfaacd1ef5a0)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:36.311559 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35541d46-6f42-4a6a-86a7-f371a0de385e None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:36.313723 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 200/392] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Aug 24 17:27:36 2022] GET /identity/v3/projects/cc1182fc1781499884cebfaacd1ef5a0 => generated 298 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:27:36.315006 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f565ab76-e7d8-44ed-b3ba-7ecf554e63cd None None] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:36.315356 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f565ab76-e7d8-44ed-b3ba-7ecf554e63cd None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:36.315699 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f565ab76-e7d8-44ed-b3ba-7ecf554e63cd None None] PATH_INFO: `/` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:36.317293 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 201/393] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:27:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:27:36.322393 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f565ab76-e7d8-44ed-b3ba-7ecf554e63cd None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:36.402814 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:36.462240 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99af8d4-b10d-427d-9dea-4ef5fd11fb28 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:36.484813 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f565ab76-e7d8-44ed-b3ba-7ecf554e63cd None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:36.509414 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6645ae26-bced-4386-81b2-c94a93fb42de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:36.510470 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6645ae26-bced-4386-81b2-c94a93fb42de None admin] REQUEST_METHOD: `PUT` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:36.510752 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6645ae26-bced-4386-81b2-c94a93fb42de None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:36.511031 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6645ae26-bced-4386-81b2-c94a93fb42de None admin] PATH_INFO: `/v3/projects/cc1182fc1781499884cebfaacd1ef5a0/groups/cf46de7dc4bf44cb96323a2d60a4be51/roles/02476a5b82eb4b188c18a5d706a3ef68` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:36.513135 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6645ae26-bced-4386-81b2-c94a93fb42de None admin] RBAC: Authorizing `identity:create_grant(project_id=cc1182fc1781499884cebfaacd1ef5a0, group_id=cf46de7dc4bf44cb96323a2d60a4be51, role_id=02476a5b82eb4b188c18a5d706a3ef68)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:36.515107 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6645ae26-bced-4386-81b2-c94a93fb42de None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:36.526882 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:36.530755 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0f9acd-988e-429a-a710-f7d685a5e1af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:36.531694 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0f9acd-988e-429a-a710-f7d685a5e1af None admin] REQUEST_METHOD: `PUT` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:36.532843 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0f9acd-988e-429a-a710-f7d685a5e1af None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:36.534483 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0f9acd-988e-429a-a710-f7d685a5e1af None admin] PATH_INFO: `/v3/projects/cc1182fc1781499884cebfaacd1ef5a0/groups/cf46de7dc4bf44cb96323a2d60a4be51/roles/148f135e4a134697a032be04bc9faa4c` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:36.536639 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 193/394] 10.222.0.45 () {58 vars in 1442 bytes} [Wed Aug 24 17:27:36 2022] PUT /identity/v3/projects/cc1182fc1781499884cebfaacd1ef5a0/groups/cf46de7dc4bf44cb96323a2d60a4be51/roles/02476a5b82eb4b188c18a5d706a3ef68 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:27:36.538247 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e393d404-9f6e-4d78-a042-5dfa1de299ee None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:36.538746 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e393d404-9f6e-4d78-a042-5dfa1de299ee None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:36.539998 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e393d404-9f6e-4d78-a042-5dfa1de299ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:36.544378 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0f9acd-988e-429a-a710-f7d685a5e1af None admin] RBAC: Authorizing `identity:create_grant(project_id=cc1182fc1781499884cebfaacd1ef5a0, group_id=cf46de7dc4bf44cb96323a2d60a4be51, role_id=148f135e4a134697a032be04bc9faa4c)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:36.546290 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0f9acd-988e-429a-a710-f7d685a5e1af None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:36.569758 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 202/395] 10.222.0.45 () {58 vars in 1442 bytes} [Wed Aug 24 17:27:36 2022] PUT /identity/v3/projects/cc1182fc1781499884cebfaacd1ef5a0/groups/cf46de7dc4bf44cb96323a2d60a4be51/roles/148f135e4a134697a032be04bc9faa4c => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:27:36.576501 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-e393d404-9f6e-4d78-a042-5dfa1de299ee None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:27:36.578198 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-e393d404-9f6e-4d78-a042-5dfa1de299ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:27:36.660381 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 194/396] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:27:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:27:37.010534 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0f9acd-988e-429a-a710-f7d685a5e1af None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:37.126973 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527f28c2-9de9-4681-bdd9-962126466d83 None None] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:37.127335 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527f28c2-9de9-4681-bdd9-962126466d83 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:37.127547 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527f28c2-9de9-4681-bdd9-962126466d83 None None] PATH_INFO: `/` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:37.129194 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 195/397] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:27:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:27:37.135639 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f734b1e3-7686-4903-ad5c-c7854e68e7b5 None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:37.135926 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f734b1e3-7686-4903-ad5c-c7854e68e7b5 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:37.136209 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f734b1e3-7686-4903-ad5c-c7854e68e7b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:37.136795 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0f9acd-988e-429a-a710-f7d685a5e1af None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:37.152124 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:37.162839 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-f734b1e3-7686-4903-ad5c-c7854e68e7b5 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:27:37.164169 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-f734b1e3-7686-4903-ad5c-c7854e68e7b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:27:37.195152 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a657463-89b0-4459-abd4-16a8008a6c9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:37.195796 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a657463-89b0-4459-abd4-16a8008a6c9b None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:37.196073 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a657463-89b0-4459-abd4-16a8008a6c9b None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:37.196342 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a657463-89b0-4459-abd4-16a8008a6c9b None admin] PATH_INFO: `/v3/roles/member` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:37.225481 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a657463-89b0-4459-abd4-16a8008a6c9b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:27:37.231057 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application [None req-1a657463-89b0-4459-abd4-16a8008a6c9b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 17:27:37.231057 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:27:37.231057 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:27:37.231057 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:27:37.231057 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:27:37.231057 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:27:37.231057 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:27:37.231057 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:27:37.231057 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:27:37.231057 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:27:37.231057 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:27:37.231057 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:27:37.231057 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:27:37.231057 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:27:37.231057 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:27:37.231057 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:27:37.231937 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:27:37.231937 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:27:37.231937 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:27:37.231937 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:27:37.231937 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:27:37.231937 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:27:37.231937 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:27:37.231937 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:27:37.231937 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:27:37.231937 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:27:37.231937 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:27:37.231937 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application with Lock( Aug 24 17:27:37.231937 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:27:37.231937 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:27:37.231937 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:27:37.231937 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:27:37.232763 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:27:37.232763 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:27:37.232763 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:27:37.232763 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:27:37.232763 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:27:37.232763 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:27:37.232763 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:27:37.232763 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:27:37.232763 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:27:37.232763 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:27:37.232763 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 17:27:37.232763 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Aug 24 17:27:37.232763 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 203/398] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Aug 24 17:27:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:27:37.238318 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a657463-89b0-4459-abd4-16a8008a6c9b None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:37.243280 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 196/399] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:27:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 17:27:37.364784 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a657463-89b0-4459-abd4-16a8008a6c9b None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:37.406175 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1f36de-bb4c-4a0b-aaa1-eb0faca497b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:37.406805 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1f36de-bb4c-4a0b-aaa1-eb0faca497b4 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:37.407089 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1f36de-bb4c-4a0b-aaa1-eb0faca497b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:37.407358 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1f36de-bb4c-4a0b-aaa1-eb0faca497b4 None admin] PATH_INFO: `/v3/roles` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:37.408033 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1f36de-bb4c-4a0b-aaa1-eb0faca497b4 None admin] RBAC: Adding query filter params (name=member) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:27:37.408337 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1f36de-bb4c-4a0b-aaa1-eb0faca497b4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:37.409745 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1f36de-bb4c-4a0b-aaa1-eb0faca497b4 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:37.422523 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 204/400] 10.222.0.45 () {56 vars in 1101 bytes} [Wed Aug 24 17:27:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:27:37.428355 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f734b1e3-7686-4903-ad5c-c7854e68e7b5 None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:37.519237 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:37.571138 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f734b1e3-7686-4903-ad5c-c7854e68e7b5 None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:37.610799 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d32887-addc-4184-a2d7-1bf89ff3a57f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:37.611469 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d32887-addc-4184-a2d7-1bf89ff3a57f None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:37.611702 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d32887-addc-4184-a2d7-1bf89ff3a57f None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:37.611970 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d32887-addc-4184-a2d7-1bf89ff3a57f None admin] PATH_INFO: `/v3/groups/cf46de7dc4bf44cb96323a2d60a4be51` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:37.613087 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d32887-addc-4184-a2d7-1bf89ff3a57f None admin] RBAC: Authorizing `identity:get_group(group_id=cf46de7dc4bf44cb96323a2d60a4be51)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:37.614530 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d32887-addc-4184-a2d7-1bf89ff3a57f None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:37.616417 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 197/401] 10.222.0.45 () {56 vars in 1180 bytes} [Wed Aug 24 17:27:37 2022] GET /identity/v3/groups/cf46de7dc4bf44cb96323a2d60a4be51 => generated 226 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:27:37.621919 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1f36de-bb4c-4a0b-aaa1-eb0faca497b4 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:37.659528 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d32887-addc-4184-a2d7-1bf89ff3a57f None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:37.667839 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:37.757347 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1f36de-bb4c-4a0b-aaa1-eb0faca497b4 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:37.805677 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8e3f34-c9ac-480b-9b9e-4cdd06a9ed6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:37.806593 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8e3f34-c9ac-480b-9b9e-4cdd06a9ed6d None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:37.807639 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8e3f34-c9ac-480b-9b9e-4cdd06a9ed6d None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:37.808838 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8e3f34-c9ac-480b-9b9e-4cdd06a9ed6d None admin] PATH_INFO: `/v3/projects/3a29643b449b4fc1aa71802d8db7c0d2` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:37.814257 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8e3f34-c9ac-480b-9b9e-4cdd06a9ed6d None admin] RBAC: Authorizing `identity:get_project(project_id=3a29643b449b4fc1aa71802d8db7c0d2)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:37.815926 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8e3f34-c9ac-480b-9b9e-4cdd06a9ed6d None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:37.817809 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 205/402] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Aug 24 17:27:37 2022] GET /identity/v3/projects/3a29643b449b4fc1aa71802d8db7c0d2 => generated 302 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:27:37.826208 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8e3f34-c9ac-480b-9b9e-4cdd06a9ed6d None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:37.870685 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d32887-addc-4184-a2d7-1bf89ff3a57f None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:37.924998 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:37.938376 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ebd425-1235-4502-8418-3dd6b3a1f8da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:37.939013 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ebd425-1235-4502-8418-3dd6b3a1f8da None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:37.939288 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ebd425-1235-4502-8418-3dd6b3a1f8da None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:37.939555 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ebd425-1235-4502-8418-3dd6b3a1f8da None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:37.965377 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ebd425-1235-4502-8418-3dd6b3a1f8da None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:27:37.970127 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application [None req-c0ebd425-1235-4502-8418-3dd6b3a1f8da None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 17:27:37.970127 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:27:37.970127 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:27:37.970127 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:27:37.970127 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:27:37.970127 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:27:37.970127 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:27:37.970127 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:27:37.970127 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:27:37.970127 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:27:37.970127 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:27:37.970127 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:27:37.970127 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:27:37.970127 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:27:37.970127 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:27:37.970127 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:27:37.971030 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:27:37.971030 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:27:37.971030 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:27:37.971030 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:27:37.971030 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:27:37.971030 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:27:37.971030 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:27:37.971030 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:27:37.971030 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:27:37.971030 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:27:37.971030 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:27:37.971030 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application with Lock( Aug 24 17:27:37.971030 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:27:37.971030 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:27:37.971030 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:27:37.971030 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:27:37.971805 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:27:37.971805 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:27:37.971805 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:27:37.971805 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:27:37.971805 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:27:37.971805 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:27:37.971805 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:27:37.971805 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:27:37.971805 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:27:37.971805 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:27:37.971805 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 17:27:37.971805 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Aug 24 17:27:37.979126 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 198/403] 10.222.0.45 () {56 vars in 1114 bytes} [Wed Aug 24 17:27:37 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 320 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:27:37.980018 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ebd425-1235-4502-8418-3dd6b3a1f8da None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:38.015018 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:38.024235 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8e3f34-c9ac-480b-9b9e-4cdd06a9ed6d None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:38.082872 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a3c23d-334c-47f6-85e4-98c9cbad38a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:38.083291 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a3c23d-334c-47f6-85e4-98c9cbad38a2 None admin] REQUEST_METHOD: `PUT` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:38.083570 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a3c23d-334c-47f6-85e4-98c9cbad38a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:38.083841 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a3c23d-334c-47f6-85e4-98c9cbad38a2 None admin] PATH_INFO: `/v3/projects/3a29643b449b4fc1aa71802d8db7c0d2/groups/cf46de7dc4bf44cb96323a2d60a4be51/roles/02476a5b82eb4b188c18a5d706a3ef68` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:38.089898 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a3c23d-334c-47f6-85e4-98c9cbad38a2 None admin] RBAC: Authorizing `identity:create_grant(project_id=3a29643b449b4fc1aa71802d8db7c0d2, group_id=cf46de7dc4bf44cb96323a2d60a4be51, role_id=02476a5b82eb4b188c18a5d706a3ef68)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:38.091348 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a3c23d-334c-47f6-85e4-98c9cbad38a2 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:38.109970 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ebd425-1235-4502-8418-3dd6b3a1f8da None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:38.113614 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 206/404] 10.222.0.45 () {58 vars in 1442 bytes} [Wed Aug 24 17:27:37 2022] PUT /identity/v3/projects/3a29643b449b4fc1aa71802d8db7c0d2/groups/cf46de7dc4bf44cb96323a2d60a4be51/roles/02476a5b82eb4b188c18a5d706a3ef68 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:27:38.155808 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bcadfa-0c25-4d17-9a75-d91798fa9856 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:38.156511 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bcadfa-0c25-4d17-9a75-d91798fa9856 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:38.156816 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bcadfa-0c25-4d17-9a75-d91798fa9856 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:38.157093 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bcadfa-0c25-4d17-9a75-d91798fa9856 None admin] PATH_INFO: `/v3/roles` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:38.157881 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bcadfa-0c25-4d17-9a75-d91798fa9856 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:27:38.158186 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bcadfa-0c25-4d17-9a75-d91798fa9856 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:38.159605 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bcadfa-0c25-4d17-9a75-d91798fa9856 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:38.172901 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 199/405] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Aug 24 17:27:37 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:27:38.174714 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a3c23d-334c-47f6-85e4-98c9cbad38a2 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:38.300803 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a3c23d-334c-47f6-85e4-98c9cbad38a2 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:38.337262 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01011a00-e4c4-4da7-beb8-a4104b15992f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:38.338311 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01011a00-e4c4-4da7-beb8-a4104b15992f None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:38.338814 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01011a00-e4c4-4da7-beb8-a4104b15992f None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:38.339279 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01011a00-e4c4-4da7-beb8-a4104b15992f None admin] PATH_INFO: `/v3/groups/cf46de7dc4bf44cb96323a2d60a4be51` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:38.340705 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01011a00-e4c4-4da7-beb8-a4104b15992f None admin] RBAC: Authorizing `identity:get_group(group_id=cf46de7dc4bf44cb96323a2d60a4be51)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:38.342361 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01011a00-e4c4-4da7-beb8-a4104b15992f None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:38.344171 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 207/406] 10.222.0.45 () {56 vars in 1180 bytes} [Wed Aug 24 17:27:38 2022] GET /identity/v3/groups/cf46de7dc4bf44cb96323a2d60a4be51 => generated 226 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:27:38.352589 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bcadfa-0c25-4d17-9a75-d91798fa9856 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:38.429002 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:38.483405 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bcadfa-0c25-4d17-9a75-d91798fa9856 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:38.523754 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf4c144-7c09-440e-8753-b5a70f3814ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:38.524482 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf4c144-7c09-440e-8753-b5a70f3814ad None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:38.524707 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf4c144-7c09-440e-8753-b5a70f3814ad None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:38.524980 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf4c144-7c09-440e-8753-b5a70f3814ad None admin] PATH_INFO: `/v3/projects/3a29643b449b4fc1aa71802d8db7c0d2` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:38.526199 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf4c144-7c09-440e-8753-b5a70f3814ad None admin] RBAC: Authorizing `identity:get_project(project_id=3a29643b449b4fc1aa71802d8db7c0d2)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:38.527693 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf4c144-7c09-440e-8753-b5a70f3814ad None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:38.529445 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 200/407] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Aug 24 17:27:38 2022] GET /identity/v3/projects/3a29643b449b4fc1aa71802d8db7c0d2 => generated 302 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:27:38.535387 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01011a00-e4c4-4da7-beb8-a4104b15992f None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:38.580923 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:38.671516 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01011a00-e4c4-4da7-beb8-a4104b15992f None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:38.717354 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcd556f-22f8-4af2-a307-7e3956d99413 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:38.717954 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcd556f-22f8-4af2-a307-7e3956d99413 None admin] REQUEST_METHOD: `PUT` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:38.718236 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcd556f-22f8-4af2-a307-7e3956d99413 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:38.718508 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcd556f-22f8-4af2-a307-7e3956d99413 None admin] PATH_INFO: `/v3/projects/3a29643b449b4fc1aa71802d8db7c0d2/groups/cf46de7dc4bf44cb96323a2d60a4be51/roles/148f135e4a134697a032be04bc9faa4c` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:38.720855 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcd556f-22f8-4af2-a307-7e3956d99413 None admin] RBAC: Authorizing `identity:create_grant(project_id=3a29643b449b4fc1aa71802d8db7c0d2, group_id=cf46de7dc4bf44cb96323a2d60a4be51, role_id=148f135e4a134697a032be04bc9faa4c)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:38.722302 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcd556f-22f8-4af2-a307-7e3956d99413 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:38.736203 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 208/408] 10.222.0.45 () {58 vars in 1442 bytes} [Wed Aug 24 17:27:38 2022] PUT /identity/v3/projects/3a29643b449b4fc1aa71802d8db7c0d2/groups/cf46de7dc4bf44cb96323a2d60a4be51/roles/148f135e4a134697a032be04bc9faa4c => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:27:38.751143 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7411bfb-72bb-47ab-bfaf-8d8a2386c9f1 None None] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:38.751829 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7411bfb-72bb-47ab-bfaf-8d8a2386c9f1 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:38.752354 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7411bfb-72bb-47ab-bfaf-8d8a2386c9f1 None None] PATH_INFO: `/` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:38.754430 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 201/409] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:27:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:27:38.761475 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbbebeb-0705-4226-b1b7-497a4499385c None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:38.761832 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbbebeb-0705-4226-b1b7-497a4499385c None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:38.762040 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbbebeb-0705-4226-b1b7-497a4499385c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:38.788599 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-dcbbebeb-0705-4226-b1b7-497a4499385c None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:27:38.789874 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-dcbbebeb-0705-4226-b1b7-497a4499385c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:27:38.882170 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 209/410] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:27:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:27:39.103487 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9fde31-5e61-431b-8376-3f3b7d9ef7d7 None None] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:39.104600 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9fde31-5e61-431b-8376-3f3b7d9ef7d7 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:39.104779 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9fde31-5e61-431b-8376-3f3b7d9ef7d7 None None] PATH_INFO: `/` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:39.106386 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 202/411] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:27:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:27:39.111296 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdab25c2-f351-4347-93f4-747221d352fe None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:39.111582 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdab25c2-f351-4347-93f4-747221d352fe None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:39.111856 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdab25c2-f351-4347-93f4-747221d352fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:39.141565 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-fdab25c2-f351-4347-93f4-747221d352fe None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:27:39.142752 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-fdab25c2-f351-4347-93f4-747221d352fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:27:39.154899 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:39.181855 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9fde31-5e61-431b-8376-3f3b7d9ef7d7 None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:39.232965 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 210/412] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:27:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:27:39.248710 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f86560d-210e-46fd-97c6-259cf0225780 None None] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:39.249000 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f86560d-210e-46fd-97c6-259cf0225780 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:39.249275 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f86560d-210e-46fd-97c6-259cf0225780 None None] PATH_INFO: `/` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:39.250587 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 211/413] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:27:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 17:27:39.266762 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e00bc3-13df-48d3-bca4-30ef0bb6f06b None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:39.267037 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e00bc3-13df-48d3-bca4-30ef0bb6f06b None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:39.267318 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e00bc3-13df-48d3-bca4-30ef0bb6f06b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:39.299792 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-a0e00bc3-13df-48d3-bca4-30ef0bb6f06b None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:27:39.302528 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-a0e00bc3-13df-48d3-bca4-30ef0bb6f06b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:27:39.316962 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9fde31-5e61-431b-8376-3f3b7d9ef7d7 None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:39.338399 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:39.367134 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be80405-4904-44ee-99e0-3644a535ede5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:39.369017 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be80405-4904-44ee-99e0-3644a535ede5 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:39.371228 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be80405-4904-44ee-99e0-3644a535ede5 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:39.371702 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be80405-4904-44ee-99e0-3644a535ede5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:39.392681 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be80405-4904-44ee-99e0-3644a535ede5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:27:39.399538 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application [None req-1be80405-4904-44ee-99e0-3644a535ede5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:27:39.399538 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:27:39.399538 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:27:39.399538 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:27:39.399538 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:27:39.399538 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:27:39.399538 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:27:39.399538 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:27:39.399538 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:27:39.399538 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:27:39.399538 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:27:39.399538 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:27:39.399538 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:27:39.399538 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:27:39.399538 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:27:39.399538 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:27:39.400534 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:27:39.400534 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:27:39.400534 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:27:39.400534 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:27:39.400534 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:27:39.400534 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:27:39.400534 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:27:39.400534 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:27:39.400534 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:27:39.400534 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:27:39.400534 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:27:39.400534 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application with Lock( Aug 24 17:27:39.400534 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:27:39.400534 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:27:39.400534 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:27:39.400534 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:27:39.401940 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:27:39.401940 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:27:39.401940 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:27:39.401940 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:27:39.401940 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:27:39.401940 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:27:39.401940 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:27:39.401940 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:27:39.401940 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:27:39.401940 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:27:39.401940 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:27:39.401940 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Aug 24 17:27:39.409070 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 203/414] 10.222.0.45 () {56 vars in 1096 bytes} [Wed Aug 24 17:27:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:27:39.411151 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be80405-4904-44ee-99e0-3644a535ede5 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:39.437165 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 212/415] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:27:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 17:27:39.542347 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be80405-4904-44ee-99e0-3644a535ede5 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:39.584858 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e00bc3-13df-48d3-bca4-30ef0bb6f06b None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:39.586721 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca101746-c468-42b8-aeb6-cf1f719b8aa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:39.587531 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca101746-c468-42b8-aeb6-cf1f719b8aa6 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:39.587962 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca101746-c468-42b8-aeb6-cf1f719b8aa6 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:39.588935 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca101746-c468-42b8-aeb6-cf1f719b8aa6 None admin] PATH_INFO: `/v3/roles` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:39.589775 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca101746-c468-42b8-aeb6-cf1f719b8aa6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:27:39.590229 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca101746-c468-42b8-aeb6-cf1f719b8aa6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:39.591803 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca101746-c468-42b8-aeb6-cf1f719b8aa6 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:39.612266 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 204/416] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Aug 24 17:27:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:27:39.617925 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca101746-c468-42b8-aeb6-cf1f719b8aa6 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:39.693530 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:39.729016 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:39.754850 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e00bc3-13df-48d3-bca4-30ef0bb6f06b None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:39.797220 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc22cf3-0599-40f1-bb9c-01f37e6b1a5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:39.797463 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc22cf3-0599-40f1-bb9c-01f37e6b1a5f None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:39.797546 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc22cf3-0599-40f1-bb9c-01f37e6b1a5f None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:39.798851 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc22cf3-0599-40f1-bb9c-01f37e6b1a5f None admin] PATH_INFO: `/v3/roles/member` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:39.801724 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca101746-c468-42b8-aeb6-cf1f719b8aa6 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:39.808012 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc22cf3-0599-40f1-bb9c-01f37e6b1a5f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:27:39.812880 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application [None req-6bc22cf3-0599-40f1-bb9c-01f37e6b1a5f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 17:27:39.812880 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:27:39.812880 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:27:39.812880 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:27:39.812880 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:27:39.812880 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:27:39.812880 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:27:39.812880 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:27:39.812880 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:27:39.812880 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:27:39.812880 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:27:39.812880 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:27:39.812880 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:27:39.812880 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:27:39.812880 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:27:39.812880 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:27:39.813838 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:27:39.813838 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:27:39.813838 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:27:39.813838 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:27:39.813838 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:27:39.813838 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:27:39.813838 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:27:39.813838 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:27:39.813838 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:27:39.813838 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:27:39.813838 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:27:39.813838 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application with Lock( Aug 24 17:27:39.813838 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:27:39.813838 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:27:39.813838 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:27:39.813838 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:27:39.814640 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:27:39.814640 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:27:39.814640 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:27:39.814640 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:27:39.814640 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:27:39.814640 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:27:39.814640 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:27:39.814640 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:27:39.814640 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:27:39.814640 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:27:39.814640 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 17:27:39.814640 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Aug 24 17:27:39.814640 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 213/417] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Aug 24 17:27:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 17:27:39.815307 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc22cf3-0599-40f1-bb9c-01f37e6b1a5f None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:39.839126 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dc21ad-6161-4888-b9f5-a03d483a66cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:39.839859 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dc21ad-6161-4888-b9f5-a03d483a66cb None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:39.840143 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dc21ad-6161-4888-b9f5-a03d483a66cb None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:39.840512 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dc21ad-6161-4888-b9f5-a03d483a66cb None admin] PATH_INFO: `/v3/groups/5877fa6c407248d4b6f9fc1a15676f44` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:39.841651 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6dc21ad-6161-4888-b9f5-a03d483a66cb None admin] RBAC: Authorizing `identity:get_group(group_id=5877fa6c407248d4b6f9fc1a15676f44)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:39.843221 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6dc21ad-6161-4888-b9f5-a03d483a66cb None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:39.845221 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 205/418] 10.222.0.45 () {56 vars in 1180 bytes} [Wed Aug 24 17:27:39 2022] GET /identity/v3/groups/5877fa6c407248d4b6f9fc1a15676f44 => generated 229 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:27:39.846278 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dc21ad-6161-4888-b9f5-a03d483a66cb None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:39.914740 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc22cf3-0599-40f1-bb9c-01f37e6b1a5f None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:39.949621 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5179fea4-d473-4c32-8f6d-f1a7f7ecd852 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:39.950354 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5179fea4-d473-4c32-8f6d-f1a7f7ecd852 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:39.950631 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5179fea4-d473-4c32-8f6d-f1a7f7ecd852 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:39.950898 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5179fea4-d473-4c32-8f6d-f1a7f7ecd852 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:39.953775 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:39.958517 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dc21ad-6161-4888-b9f5-a03d483a66cb None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:39.962606 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5179fea4-d473-4c32-8f6d-f1a7f7ecd852 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:27:39.967124 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:39.968660 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application [None req-5179fea4-d473-4c32-8f6d-f1a7f7ecd852 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 17:27:39.968660 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:27:39.968660 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:27:39.968660 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:27:39.968660 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:27:39.968660 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:27:39.968660 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:27:39.968660 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:27:39.968660 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:27:39.968660 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:27:39.968660 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:27:39.968660 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:27:39.968660 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:27:39.968660 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:27:39.968660 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:27:39.968660 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:27:39.969763 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:27:39.969763 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:27:39.969763 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:27:39.969763 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:27:39.969763 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:27:39.969763 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:27:39.969763 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:27:39.969763 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:27:39.969763 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:27:39.969763 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:27:39.969763 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:27:39.969763 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application with Lock( Aug 24 17:27:39.969763 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:27:39.969763 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:27:39.969763 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:27:39.969763 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:27:39.970691 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:27:39.970691 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:27:39.970691 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:27:39.970691 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:27:39.970691 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:27:39.970691 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:27:39.970691 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:27:39.970691 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:27:39.970691 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:27:39.970691 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:27:39.970691 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 17:27:39.970691 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Aug 24 17:27:39.972778 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 214/419] 10.222.0.45 () {56 vars in 1114 bytes} [Wed Aug 24 17:27:39 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:27:39.973704 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5179fea4-d473-4c32-8f6d-f1a7f7ecd852 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:40.012809 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64f5bf9-81b3-434c-8e38-bbb2b0543269 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:40.013856 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64f5bf9-81b3-434c-8e38-bbb2b0543269 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:40.014152 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64f5bf9-81b3-434c-8e38-bbb2b0543269 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:40.014432 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64f5bf9-81b3-434c-8e38-bbb2b0543269 None admin] PATH_INFO: `/v3/roles` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:40.015434 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64f5bf9-81b3-434c-8e38-bbb2b0543269 None admin] RBAC: Adding query filter params (name=member) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:27:40.015790 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64f5bf9-81b3-434c-8e38-bbb2b0543269 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:40.017838 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64f5bf9-81b3-434c-8e38-bbb2b0543269 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:40.029037 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 206/420] 10.222.0.45 () {56 vars in 1101 bytes} [Wed Aug 24 17:27:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:27:40.030031 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64f5bf9-81b3-434c-8e38-bbb2b0543269 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:40.095775 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5179fea4-d473-4c32-8f6d-f1a7f7ecd852 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:40.133140 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8489a151-0cbd-4c42-82ec-4a32181f7fcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:40.133888 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8489a151-0cbd-4c42-82ec-4a32181f7fcd None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:40.134237 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8489a151-0cbd-4c42-82ec-4a32181f7fcd None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:40.134684 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8489a151-0cbd-4c42-82ec-4a32181f7fcd None admin] PATH_INFO: `/v3/projects/1e847003e2ef47df81d36ac4caad47f0` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:40.136070 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8489a151-0cbd-4c42-82ec-4a32181f7fcd None admin] RBAC: Authorizing `identity:get_project(project_id=1e847003e2ef47df81d36ac4caad47f0)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:40.137700 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8489a151-0cbd-4c42-82ec-4a32181f7fcd None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:40.139928 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 215/421] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Aug 24 17:27:39 2022] GET /identity/v3/projects/1e847003e2ef47df81d36ac4caad47f0 => generated 344 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:27:40.140828 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8489a151-0cbd-4c42-82ec-4a32181f7fcd None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:40.159096 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64f5bf9-81b3-434c-8e38-bbb2b0543269 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:40.205764 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1b1700-97cb-42c1-8822-74fd7ebd6767 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:40.206612 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1b1700-97cb-42c1-8822-74fd7ebd6767 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:40.207154 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1b1700-97cb-42c1-8822-74fd7ebd6767 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:40.207551 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1b1700-97cb-42c1-8822-74fd7ebd6767 None admin] PATH_INFO: `/v3/roles` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:40.208235 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1b1700-97cb-42c1-8822-74fd7ebd6767 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:27:40.208728 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1b1700-97cb-42c1-8822-74fd7ebd6767 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:40.210793 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1b1700-97cb-42c1-8822-74fd7ebd6767 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:40.221134 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 207/422] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Aug 24 17:27:40 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:27:40.221928 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1b1700-97cb-42c1-8822-74fd7ebd6767 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:40.224901 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:40.255278 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8489a151-0cbd-4c42-82ec-4a32181f7fcd None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:40.269692 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:40.295220 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc5d148-0687-46be-897b-320b96cccb16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:40.296350 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc5d148-0687-46be-897b-320b96cccb16 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:40.296873 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc5d148-0687-46be-897b-320b96cccb16 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:40.297359 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc5d148-0687-46be-897b-320b96cccb16 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:40.361731 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1b1700-97cb-42c1-8822-74fd7ebd6767 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:40.375441 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc5d148-0687-46be-897b-320b96cccb16 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:40.377203 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc5d148-0687-46be-897b-320b96cccb16 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:40.400188 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a14f9d-95e6-46e8-a462-ac1c05420080 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:40.401246 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a14f9d-95e6-46e8-a462-ac1c05420080 None admin] REQUEST_METHOD: `PUT` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:40.401754 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a14f9d-95e6-46e8-a462-ac1c05420080 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:40.402240 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a14f9d-95e6-46e8-a462-ac1c05420080 None admin] PATH_INFO: `/v3/projects/1e847003e2ef47df81d36ac4caad47f0/groups/5877fa6c407248d4b6f9fc1a15676f44/roles/e411ad9eb96e4e8d85b59b5b700ada7b` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:40.404413 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a14f9d-95e6-46e8-a462-ac1c05420080 None admin] RBAC: Authorizing `identity:create_grant(project_id=1e847003e2ef47df81d36ac4caad47f0, group_id=5877fa6c407248d4b6f9fc1a15676f44, role_id=e411ad9eb96e4e8d85b59b5b700ada7b)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:40.406097 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a14f9d-95e6-46e8-a462-ac1c05420080 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:40.419922 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 208/423] 10.222.0.45 () {58 vars in 1442 bytes} [Wed Aug 24 17:27:40 2022] PUT /identity/v3/projects/1e847003e2ef47df81d36ac4caad47f0/groups/5877fa6c407248d4b6f9fc1a15676f44/roles/e411ad9eb96e4e8d85b59b5b700ada7b => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:27:40.421231 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a14f9d-95e6-46e8-a462-ac1c05420080 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:40.451911 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:40.499900 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 216/424] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Aug 24 17:27:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 17:27:40.509255 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc5d148-0687-46be-897b-320b96cccb16 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:40.532692 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:40.556953 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a14f9d-95e6-46e8-a462-ac1c05420080 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:40.625162 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932e5fb9-4f93-408b-97fb-20e6a89b30b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:40.626148 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932e5fb9-4f93-408b-97fb-20e6a89b30b0 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:40.626426 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932e5fb9-4f93-408b-97fb-20e6a89b30b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:40.626698 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932e5fb9-4f93-408b-97fb-20e6a89b30b0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:40.634844 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc5d148-0687-46be-897b-320b96cccb16 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:40.675242 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0568f2b3-01f1-4906-98d9-0c0de1c0f738 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:40.675999 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0568f2b3-01f1-4906-98d9-0c0de1c0f738 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:40.676351 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0568f2b3-01f1-4906-98d9-0c0de1c0f738 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:40.676761 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0568f2b3-01f1-4906-98d9-0c0de1c0f738 None admin] PATH_INFO: `/v3/projects/cc1182fc1781499884cebfaacd1ef5a0` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:40.677981 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0568f2b3-01f1-4906-98d9-0c0de1c0f738 None admin] RBAC: Authorizing `identity:get_project(project_id=cc1182fc1781499884cebfaacd1ef5a0)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:40.679501 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0568f2b3-01f1-4906-98d9-0c0de1c0f738 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:40.686259 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 217/425] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Aug 24 17:27:40 2022] GET /identity/v3/projects/cc1182fc1781499884cebfaacd1ef5a0 => generated 298 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:27:40.687772 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0568f2b3-01f1-4906-98d9-0c0de1c0f738 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:40.722135 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:40.727928 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932e5fb9-4f93-408b-97fb-20e6a89b30b0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:40.730602 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932e5fb9-4f93-408b-97fb-20e6a89b30b0 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:40.806948 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0568f2b3-01f1-4906-98d9-0c0de1c0f738 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:40.850317 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa7e566-3faa-430c-a13c-b067a6a3fb75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:40.851422 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa7e566-3faa-430c-a13c-b067a6a3fb75 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:40.851921 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa7e566-3faa-430c-a13c-b067a6a3fb75 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:40.852890 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa7e566-3faa-430c-a13c-b067a6a3fb75 None admin] PATH_INFO: `/v3/groups/cf46de7dc4bf44cb96323a2d60a4be51` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:40.855269 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa7e566-3faa-430c-a13c-b067a6a3fb75 None admin] RBAC: Authorizing `identity:get_group(group_id=cf46de7dc4bf44cb96323a2d60a4be51)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:40.855949 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:40.858960 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa7e566-3faa-430c-a13c-b067a6a3fb75 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:40.862183 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 218/426] 10.222.0.45 () {56 vars in 1180 bytes} [Wed Aug 24 17:27:40 2022] GET /identity/v3/groups/cf46de7dc4bf44cb96323a2d60a4be51 => generated 226 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:27:40.867124 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91363b08-78b8-43fe-abf2-a41e8c971295 None None] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:40.867124 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91363b08-78b8-43fe-abf2-a41e8c971295 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:40.867124 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91363b08-78b8-43fe-abf2-a41e8c971295 None None] PATH_INFO: `/` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:40.868315 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 219/427] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:27:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:27:40.870226 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91363b08-78b8-43fe-abf2-a41e8c971295 None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:40.877312 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 209/428] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Aug 24 17:27:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 457 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 17:27:40.878547 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb29372d-425e-41f7-908e-1fd8a5ba246e None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:40.878831 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb29372d-425e-41f7-908e-1fd8a5ba246e None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:40.879107 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb29372d-425e-41f7-908e-1fd8a5ba246e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:40.902884 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-cb29372d-425e-41f7-908e-1fd8a5ba246e None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:27:40.904084 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-cb29372d-425e-41f7-908e-1fd8a5ba246e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:27:40.987784 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 210/429] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:27:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:27:40.987784 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb29372d-425e-41f7-908e-1fd8a5ba246e None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:41.017052 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:41.023338 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91363b08-78b8-43fe-abf2-a41e8c971295 None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:41.081708 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269ee95b-bdaa-4705-b40d-27af3306abc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:41.082231 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269ee95b-bdaa-4705-b40d-27af3306abc9 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:41.082506 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269ee95b-bdaa-4705-b40d-27af3306abc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:41.082777 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269ee95b-bdaa-4705-b40d-27af3306abc9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:41.083493 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269ee95b-bdaa-4705-b40d-27af3306abc9 None admin] RBAC: Adding query filter params (group.id=cf46de7dc4bf44cb96323a2d60a4be51, role.id=02476a5b82eb4b188c18a5d706a3ef68, scope.project.id=cc1182fc1781499884cebfaacd1ef5a0) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:27:41.084204 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269ee95b-bdaa-4705-b40d-27af3306abc9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:41.086082 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269ee95b-bdaa-4705-b40d-27af3306abc9 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:41.093497 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 220/430] 10.222.0.45 () {56 vars in 1376 bytes} [Wed Aug 24 17:27:40 2022] GET /identity/v3/role_assignments?group.id=cf46de7dc4bf44cb96323a2d60a4be51&scope.project.id=cc1182fc1781499884cebfaacd1ef5a0&role.id=02476a5b82eb4b188c18a5d706a3ef68 => generated 611 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:27:41.095213 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a29d8f-f071-4e75-94f5-5bf6db8cb56e None None] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:41.095495 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a29d8f-f071-4e75-94f5-5bf6db8cb56e None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:41.095771 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a29d8f-f071-4e75-94f5-5bf6db8cb56e None None] PATH_INFO: `/` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:41.100724 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 221/431] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:27:41 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:27:41.104415 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56a2d69-e7fb-4d4a-b848-c0c0f1efd3bc None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:41.104745 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56a2d69-e7fb-4d4a-b848-c0c0f1efd3bc None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:41.105023 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56a2d69-e7fb-4d4a-b848-c0c0f1efd3bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:41.116689 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb29372d-425e-41f7-908e-1fd8a5ba246e None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:41.124567 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-d56a2d69-e7fb-4d4a-b848-c0c0f1efd3bc None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:27:41.125829 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-d56a2d69-e7fb-4d4a-b848-c0c0f1efd3bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:27:41.141010 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:41.155336 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56de105a-c40f-411d-a158-f4123622042f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:41.155978 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56de105a-c40f-411d-a158-f4123622042f None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:41.156262 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56de105a-c40f-411d-a158-f4123622042f None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:41.156534 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56de105a-c40f-411d-a158-f4123622042f None admin] PATH_INFO: `/v3/projects/cc1182fc1781499884cebfaacd1ef5a0` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:41.157899 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56de105a-c40f-411d-a158-f4123622042f None admin] RBAC: Authorizing `identity:get_project(project_id=cc1182fc1781499884cebfaacd1ef5a0)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:41.159348 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56de105a-c40f-411d-a158-f4123622042f None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:41.161575 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 211/432] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Aug 24 17:27:40 2022] GET /identity/v3/projects/cc1182fc1781499884cebfaacd1ef5a0 => generated 298 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:27:41.167292 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56de105a-c40f-411d-a158-f4123622042f None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:41.202793 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 222/433] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:27:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:27:41.321894 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56de105a-c40f-411d-a158-f4123622042f None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:41.329581 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56a2d69-e7fb-4d4a-b848-c0c0f1efd3bc None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:41.386126 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57e6658-889d-4ad1-9e95-8ed15243a58c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:41.386695 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57e6658-889d-4ad1-9e95-8ed15243a58c None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:41.386979 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57e6658-889d-4ad1-9e95-8ed15243a58c None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:41.387249 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57e6658-889d-4ad1-9e95-8ed15243a58c None admin] PATH_INFO: `/v3/groups/cf46de7dc4bf44cb96323a2d60a4be51` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:41.388379 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57e6658-889d-4ad1-9e95-8ed15243a58c None admin] RBAC: Authorizing `identity:get_group(group_id=cf46de7dc4bf44cb96323a2d60a4be51)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:41.389897 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57e6658-889d-4ad1-9e95-8ed15243a58c None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:41.391474 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 212/434] 10.222.0.45 () {56 vars in 1180 bytes} [Wed Aug 24 17:27:41 2022] GET /identity/v3/groups/cf46de7dc4bf44cb96323a2d60a4be51 => generated 226 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:27:41.399334 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57e6658-889d-4ad1-9e95-8ed15243a58c None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:41.457098 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:41.465431 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56a2d69-e7fb-4d4a-b848-c0c0f1efd3bc None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:41.474207 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:41.514652 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06bd239-567b-4809-bc88-2a38f97cbadb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:41.515647 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06bd239-567b-4809-bc88-2a38f97cbadb None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:41.516122 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06bd239-567b-4809-bc88-2a38f97cbadb None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:41.516656 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06bd239-567b-4809-bc88-2a38f97cbadb None admin] PATH_INFO: `/v3/roles/member` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:41.529338 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06bd239-567b-4809-bc88-2a38f97cbadb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:27:41.534635 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57e6658-889d-4ad1-9e95-8ed15243a58c None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:41.535564 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application [None req-c06bd239-567b-4809-bc88-2a38f97cbadb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 17:27:41.535564 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:27:41.535564 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:27:41.535564 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:27:41.535564 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:27:41.535564 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:27:41.535564 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:27:41.535564 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:27:41.535564 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:27:41.535564 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:27:41.535564 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:27:41.535564 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:27:41.535564 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:27:41.535564 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:27:41.535564 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:27:41.535564 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:27:41.536537 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:27:41.536537 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:27:41.536537 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:27:41.536537 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:27:41.536537 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:27:41.536537 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:27:41.536537 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:27:41.536537 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:27:41.536537 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:27:41.536537 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:27:41.536537 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:27:41.536537 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application with Lock( Aug 24 17:27:41.536537 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:27:41.536537 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:27:41.536537 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:27:41.536537 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:27:41.537382 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:27:41.537382 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:27:41.537382 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:27:41.537382 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:27:41.537382 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:27:41.537382 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:27:41.537382 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:27:41.537382 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:27:41.537382 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:27:41.537382 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:27:41.537382 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 17:27:41.537382 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Aug 24 17:27:41.539676 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 223/435] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Aug 24 17:27:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 17:27:41.539676 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06bd239-567b-4809-bc88-2a38f97cbadb None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:41.591128 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da36861-3ea7-4377-9fb2-897805ce2c99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:41.591886 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da36861-3ea7-4377-9fb2-897805ce2c99 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:41.592299 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da36861-3ea7-4377-9fb2-897805ce2c99 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:41.592619 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da36861-3ea7-4377-9fb2-897805ce2c99 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:41.593620 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da36861-3ea7-4377-9fb2-897805ce2c99 None admin] RBAC: Adding query filter params (group.id=cf46de7dc4bf44cb96323a2d60a4be51, role.id=148f135e4a134697a032be04bc9faa4c, scope.project.id=cc1182fc1781499884cebfaacd1ef5a0) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:27:41.593925 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da36861-3ea7-4377-9fb2-897805ce2c99 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:41.595735 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da36861-3ea7-4377-9fb2-897805ce2c99 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:41.603392 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 213/436] 10.222.0.45 () {56 vars in 1376 bytes} [Wed Aug 24 17:27:41 2022] GET /identity/v3/role_assignments?group.id=cf46de7dc4bf44cb96323a2d60a4be51&scope.project.id=cc1182fc1781499884cebfaacd1ef5a0&role.id=148f135e4a134697a032be04bc9faa4c => generated 611 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:27:41.605590 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da36861-3ea7-4377-9fb2-897805ce2c99 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:41.669076 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06bd239-567b-4809-bc88-2a38f97cbadb None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:41.721923 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2de323-1a51-43d2-9ebf-26936499f43d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:41.723231 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2de323-1a51-43d2-9ebf-26936499f43d None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:41.723726 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2de323-1a51-43d2-9ebf-26936499f43d None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:41.724405 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2de323-1a51-43d2-9ebf-26936499f43d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:41.733706 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da36861-3ea7-4377-9fb2-897805ce2c99 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:41.746380 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2de323-1a51-43d2-9ebf-26936499f43d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:27:41.753184 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application [None req-ea2de323-1a51-43d2-9ebf-26936499f43d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 17:27:41.753184 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:27:41.753184 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:27:41.753184 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:27:41.753184 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:27:41.753184 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:27:41.753184 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:27:41.753184 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:27:41.753184 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:27:41.753184 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:27:41.753184 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:27:41.753184 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:27:41.753184 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:27:41.753184 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:27:41.753184 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:27:41.753184 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:27:41.754166 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:27:41.754166 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:27:41.754166 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:27:41.754166 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:27:41.754166 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:27:41.754166 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:27:41.754166 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:27:41.754166 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:27:41.754166 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:27:41.754166 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:27:41.754166 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:27:41.754166 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application with Lock( Aug 24 17:27:41.754166 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:27:41.754166 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:27:41.754166 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:27:41.754166 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:27:41.754940 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:27:41.754940 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:27:41.754940 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:27:41.754940 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:27:41.754940 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:27:41.754940 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:27:41.754940 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:27:41.754940 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:27:41.754940 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:27:41.754940 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:27:41.754940 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 17:27:41.754940 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Aug 24 17:27:41.757873 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 224/437] 10.222.0.45 () {56 vars in 1114 bytes} [Wed Aug 24 17:27:41 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:27:41.763128 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:41.767012 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2de323-1a51-43d2-9ebf-26936499f43d None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:41.787241 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6640b5-bab6-44c8-8846-ba8eea40371e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:41.787866 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6640b5-bab6-44c8-8846-ba8eea40371e None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:41.788141 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6640b5-bab6-44c8-8846-ba8eea40371e None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:41.788408 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6640b5-bab6-44c8-8846-ba8eea40371e None admin] PATH_INFO: `/v3/roles` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:41.789113 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6640b5-bab6-44c8-8846-ba8eea40371e None admin] RBAC: Adding query filter params (name=member) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:27:41.789414 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6640b5-bab6-44c8-8846-ba8eea40371e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:41.790799 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6640b5-bab6-44c8-8846-ba8eea40371e None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:41.799334 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:41.803244 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 214/438] 10.222.0.45 () {56 vars in 1101 bytes} [Wed Aug 24 17:27:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:27:41.810627 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6640b5-bab6-44c8-8846-ba8eea40371e None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:41.913157 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2de323-1a51-43d2-9ebf-26936499f43d None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:41.945579 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6640b5-bab6-44c8-8846-ba8eea40371e None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:41.953122 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddbb67f-25e4-4d39-bc61-5a126f455cb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:41.953749 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddbb67f-25e4-4d39-bc61-5a126f455cb4 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:41.954027 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddbb67f-25e4-4d39-bc61-5a126f455cb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:41.954297 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddbb67f-25e4-4d39-bc61-5a126f455cb4 None admin] PATH_INFO: `/v3/roles` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:41.954967 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddbb67f-25e4-4d39-bc61-5a126f455cb4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:27:41.955270 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddbb67f-25e4-4d39-bc61-5a126f455cb4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:41.957853 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddbb67f-25e4-4d39-bc61-5a126f455cb4 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:41.965736 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 225/439] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Aug 24 17:27:41 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:27:41.971135 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddbb67f-25e4-4d39-bc61-5a126f455cb4 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:41.985058 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1363d7c0-0eed-4d4d-b8b3-ecd2690c77f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:41.985686 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1363d7c0-0eed-4d4d-b8b3-ecd2690c77f6 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:41.985961 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1363d7c0-0eed-4d4d-b8b3-ecd2690c77f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:41.986230 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1363d7c0-0eed-4d4d-b8b3-ecd2690c77f6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:42.056540 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:42.076145 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:42.081191 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1363d7c0-0eed-4d4d-b8b3-ecd2690c77f6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:42.083391 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1363d7c0-0eed-4d4d-b8b3-ecd2690c77f6 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:42.108785 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddbb67f-25e4-4d39-bc61-5a126f455cb4 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:42.153028 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35e6cc0-eb29-4f26-ac1f-346b3e038016 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:42.154529 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35e6cc0-eb29-4f26-ac1f-346b3e038016 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:42.155243 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35e6cc0-eb29-4f26-ac1f-346b3e038016 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:42.155708 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35e6cc0-eb29-4f26-ac1f-346b3e038016 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:42.221950 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 215/440] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Aug 24 17:27:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 17:27:42.227391 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1363d7c0-0eed-4d4d-b8b3-ecd2690c77f6 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:42.236580 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35e6cc0-eb29-4f26-ac1f-346b3e038016 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:42.238428 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35e6cc0-eb29-4f26-ac1f-346b3e038016 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:42.315501 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:42.323952 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:42.338541 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1363d7c0-0eed-4d4d-b8b3-ecd2690c77f6 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:42.347786 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 226/441] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Aug 24 17:27:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 17:27:42.353377 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35e6cc0-eb29-4f26-ac1f-346b3e038016 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:42.376130 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea5ae93-edd8-44c3-b34d-492819f235ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:42.376898 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea5ae93-edd8-44c3-b34d-492819f235ea None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:42.377163 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea5ae93-edd8-44c3-b34d-492819f235ea None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:42.377429 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea5ae93-edd8-44c3-b34d-492819f235ea None admin] PATH_INFO: `/v3/projects/3a29643b449b4fc1aa71802d8db7c0d2` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:42.378653 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea5ae93-edd8-44c3-b34d-492819f235ea None admin] RBAC: Authorizing `identity:get_project(project_id=3a29643b449b4fc1aa71802d8db7c0d2)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:42.380188 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea5ae93-edd8-44c3-b34d-492819f235ea None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:42.386471 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 216/442] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Aug 24 17:27:42 2022] GET /identity/v3/projects/3a29643b449b4fc1aa71802d8db7c0d2 => generated 302 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:27:42.387426 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea5ae93-edd8-44c3-b34d-492819f235ea None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:42.455941 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35e6cc0-eb29-4f26-ac1f-346b3e038016 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:42.489768 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea5ae93-edd8-44c3-b34d-492819f235ea None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:42.491087 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632e89a3-dc2b-4d6c-8dc1-3293df41288a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:42.491723 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632e89a3-dc2b-4d6c-8dc1-3293df41288a None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:42.492003 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632e89a3-dc2b-4d6c-8dc1-3293df41288a None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:42.492277 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632e89a3-dc2b-4d6c-8dc1-3293df41288a None admin] PATH_INFO: `/v3/projects/3a29643b449b4fc1aa71802d8db7c0d2` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:42.493441 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632e89a3-dc2b-4d6c-8dc1-3293df41288a None admin] RBAC: Authorizing `identity:get_project(project_id=3a29643b449b4fc1aa71802d8db7c0d2)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:42.494920 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632e89a3-dc2b-4d6c-8dc1-3293df41288a None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:42.496538 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 227/443] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Aug 24 17:27:42 2022] GET /identity/v3/projects/3a29643b449b4fc1aa71802d8db7c0d2 => generated 302 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:27:42.501904 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632e89a3-dc2b-4d6c-8dc1-3293df41288a None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:42.525447 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806e1ace-dc0a-4186-8ae9-fd351945806d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:42.526617 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806e1ace-dc0a-4186-8ae9-fd351945806d None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:42.526938 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806e1ace-dc0a-4186-8ae9-fd351945806d None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:42.527216 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806e1ace-dc0a-4186-8ae9-fd351945806d None admin] PATH_INFO: `/v3/groups/cf46de7dc4bf44cb96323a2d60a4be51` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:42.528342 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806e1ace-dc0a-4186-8ae9-fd351945806d None admin] RBAC: Authorizing `identity:get_group(group_id=cf46de7dc4bf44cb96323a2d60a4be51)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:42.529827 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806e1ace-dc0a-4186-8ae9-fd351945806d None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:42.531543 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 217/444] 10.222.0.45 () {56 vars in 1180 bytes} [Wed Aug 24 17:27:42 2022] GET /identity/v3/groups/cf46de7dc4bf44cb96323a2d60a4be51 => generated 226 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:27:42.537445 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806e1ace-dc0a-4186-8ae9-fd351945806d None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:42.554287 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:42.557575 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:42.611054 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632e89a3-dc2b-4d6c-8dc1-3293df41288a None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:42.639309 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806e1ace-dc0a-4186-8ae9-fd351945806d None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:42.644287 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242f157d-0cdd-4594-8980-e674bd1b429a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:42.644934 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242f157d-0cdd-4594-8980-e674bd1b429a None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:42.645211 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242f157d-0cdd-4594-8980-e674bd1b429a None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:42.645479 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242f157d-0cdd-4594-8980-e674bd1b429a None admin] PATH_INFO: `/v3/groups/cf46de7dc4bf44cb96323a2d60a4be51` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:42.646582 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242f157d-0cdd-4594-8980-e674bd1b429a None admin] RBAC: Authorizing `identity:get_group(group_id=cf46de7dc4bf44cb96323a2d60a4be51)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:42.647998 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242f157d-0cdd-4594-8980-e674bd1b429a None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:42.649583 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 228/445] 10.222.0.45 () {56 vars in 1180 bytes} [Wed Aug 24 17:27:42 2022] GET /identity/v3/groups/cf46de7dc4bf44cb96323a2d60a4be51 => generated 226 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:27:42.651197 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad30c61-49a6-4d71-961b-8fa8a969f5ea None None] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:42.651472 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad30c61-49a6-4d71-961b-8fa8a969f5ea None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:42.651741 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad30c61-49a6-4d71-961b-8fa8a969f5ea None None] PATH_INFO: `/` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:42.652922 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 229/446] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:27:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:27:42.657469 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad30c61-49a6-4d71-961b-8fa8a969f5ea None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:42.677928 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425caef2-9027-4e33-997f-e6ea79c3b4b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:42.678533 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425caef2-9027-4e33-997f-e6ea79c3b4b3 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:42.678808 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425caef2-9027-4e33-997f-e6ea79c3b4b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:42.679079 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425caef2-9027-4e33-997f-e6ea79c3b4b3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:42.679788 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425caef2-9027-4e33-997f-e6ea79c3b4b3 None admin] RBAC: Adding query filter params (group.id=cf46de7dc4bf44cb96323a2d60a4be51, role.id=02476a5b82eb4b188c18a5d706a3ef68, scope.project.id=3a29643b449b4fc1aa71802d8db7c0d2) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:27:42.680098 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425caef2-9027-4e33-997f-e6ea79c3b4b3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:42.681551 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425caef2-9027-4e33-997f-e6ea79c3b4b3 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:42.687073 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 218/447] 10.222.0.45 () {56 vars in 1376 bytes} [Wed Aug 24 17:27:42 2022] GET /identity/v3/role_assignments?group.id=cf46de7dc4bf44cb96323a2d60a4be51&scope.project.id=3a29643b449b4fc1aa71802d8db7c0d2&role.id=02476a5b82eb4b188c18a5d706a3ef68 => generated 611 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:27:42.689828 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0332b256-2ab4-4b16-a47f-89f4fac3e1c9 None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:42.690164 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0332b256-2ab4-4b16-a47f-89f4fac3e1c9 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:42.690391 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0332b256-2ab4-4b16-a47f-89f4fac3e1c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:42.708967 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-0332b256-2ab4-4b16-a47f-89f4fac3e1c9 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:27:42.710206 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-0332b256-2ab4-4b16-a47f-89f4fac3e1c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:27:42.771291 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad30c61-49a6-4d71-961b-8fa8a969f5ea None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:42.788597 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 219/448] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:27:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:27:42.789327 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:42.817391 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad5f2d0-db23-44aa-bc43-aa4ec2f8f7b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:42.818007 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad5f2d0-db23-44aa-bc43-aa4ec2f8f7b6 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:42.818281 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad5f2d0-db23-44aa-bc43-aa4ec2f8f7b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:42.818549 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad5f2d0-db23-44aa-bc43-aa4ec2f8f7b6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:42.819283 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad5f2d0-db23-44aa-bc43-aa4ec2f8f7b6 None admin] RBAC: Adding query filter params (group.id=cf46de7dc4bf44cb96323a2d60a4be51, role.id=148f135e4a134697a032be04bc9faa4c, scope.project.id=3a29643b449b4fc1aa71802d8db7c0d2) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:27:42.819587 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad5f2d0-db23-44aa-bc43-aa4ec2f8f7b6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:42.821073 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad5f2d0-db23-44aa-bc43-aa4ec2f8f7b6 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:42.828163 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 230/449] 10.222.0.45 () {56 vars in 1376 bytes} [Wed Aug 24 17:27:42 2022] GET /identity/v3/role_assignments?group.id=cf46de7dc4bf44cb96323a2d60a4be51&scope.project.id=3a29643b449b4fc1aa71802d8db7c0d2&role.id=148f135e4a134697a032be04bc9faa4c => generated 611 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:27:43.060604 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0332b256-2ab4-4b16-a47f-89f4fac3e1c9 None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:43.077462 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:43.164803 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0332b256-2ab4-4b16-a47f-89f4fac3e1c9 None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:43.199516 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c659e9-4709-460f-9e7a-b6be53f120ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:43.200150 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c659e9-4709-460f-9e7a-b6be53f120ed None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:43.200429 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c659e9-4709-460f-9e7a-b6be53f120ed None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:43.200733 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c659e9-4709-460f-9e7a-b6be53f120ed None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:43.210872 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c659e9-4709-460f-9e7a-b6be53f120ed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:27:43.215635 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application [None req-21c659e9-4709-460f-9e7a-b6be53f120ed None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:27:43.215635 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:27:43.215635 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:27:43.215635 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:27:43.215635 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:27:43.215635 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:27:43.215635 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:27:43.215635 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:27:43.215635 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:27:43.215635 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:27:43.215635 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:27:43.215635 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:27:43.215635 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:27:43.215635 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:27:43.215635 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:27:43.215635 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:27:43.216753 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:27:43.216753 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:27:43.216753 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:27:43.216753 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:27:43.216753 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:27:43.216753 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:27:43.216753 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:27:43.216753 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:27:43.216753 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:27:43.216753 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:27:43.216753 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:27:43.216753 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application with Lock( Aug 24 17:27:43.216753 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:27:43.216753 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:27:43.216753 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:27:43.216753 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:27:43.217546 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:27:43.217546 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:27:43.217546 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:27:43.217546 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:27:43.217546 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:27:43.217546 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:27:43.217546 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:27:43.217546 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:27:43.217546 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:27:43.217546 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:27:43.217546 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:27:43.217546 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Aug 24 17:27:43.217546 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 220/450] 10.222.0.45 () {56 vars in 1096 bytes} [Wed Aug 24 17:27:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:27:43.222608 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad5f2d0-db23-44aa-bc43-aa4ec2f8f7b6 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:43.318994 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad5f2d0-db23-44aa-bc43-aa4ec2f8f7b6 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:43.351191 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54525c3e-d55a-4b4c-9f19-45b9695668cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:43.351786 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54525c3e-d55a-4b4c-9f19-45b9695668cc None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:43.352074 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54525c3e-d55a-4b4c-9f19-45b9695668cc None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:43.352350 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54525c3e-d55a-4b4c-9f19-45b9695668cc None admin] PATH_INFO: `/v3/roles` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:43.353040 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54525c3e-d55a-4b4c-9f19-45b9695668cc None admin] RBAC: Adding query filter params (name=admin) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:27:43.353346 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54525c3e-d55a-4b4c-9f19-45b9695668cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:43.355010 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54525c3e-d55a-4b4c-9f19-45b9695668cc None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:43.362759 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 231/451] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Aug 24 17:27:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:27:43.368208 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c659e9-4709-460f-9e7a-b6be53f120ed None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:43.436919 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:43.466061 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c659e9-4709-460f-9e7a-b6be53f120ed None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:43.498172 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9a7b20-5877-4c24-998d-f3a86fce6a2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:43.498761 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9a7b20-5877-4c24-998d-f3a86fce6a2c None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:43.499034 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9a7b20-5877-4c24-998d-f3a86fce6a2c None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:43.499300 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9a7b20-5877-4c24-998d-f3a86fce6a2c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:43.562841 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9a7b20-5877-4c24-998d-f3a86fce6a2c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:43.564290 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9a7b20-5877-4c24-998d-f3a86fce6a2c None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:43.628154 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:43.657978 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 221/452] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Aug 24 17:27:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 17:27:43.663823 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54525c3e-d55a-4b4c-9f19-45b9695668cc None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:43.703042 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:43.761516 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54525c3e-d55a-4b4c-9f19-45b9695668cc None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:43.793537 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf94fc70-d66a-4404-9ace-312ff8ce2fd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:43.794164 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf94fc70-d66a-4404-9ace-312ff8ce2fd7 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:43.794445 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf94fc70-d66a-4404-9ace-312ff8ce2fd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:43.794715 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf94fc70-d66a-4404-9ace-312ff8ce2fd7 None admin] PATH_INFO: `/v3/projects/1e847003e2ef47df81d36ac4caad47f0` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:43.795834 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf94fc70-d66a-4404-9ace-312ff8ce2fd7 None admin] RBAC: Authorizing `identity:get_project(project_id=1e847003e2ef47df81d36ac4caad47f0)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:43.797297 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf94fc70-d66a-4404-9ace-312ff8ce2fd7 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:43.798906 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 232/453] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Aug 24 17:27:43 2022] GET /identity/v3/projects/1e847003e2ef47df81d36ac4caad47f0 => generated 344 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:27:43.804508 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9a7b20-5877-4c24-998d-f3a86fce6a2c None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:43.898726 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9a7b20-5877-4c24-998d-f3a86fce6a2c None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:43.932204 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33131990-23cf-4db3-942e-c99456a3e1dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:43.932841 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33131990-23cf-4db3-942e-c99456a3e1dc None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:43.933120 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33131990-23cf-4db3-942e-c99456a3e1dc None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:43.933394 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33131990-23cf-4db3-942e-c99456a3e1dc None admin] PATH_INFO: `/v3/groups/5877fa6c407248d4b6f9fc1a15676f44` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:43.934482 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33131990-23cf-4db3-942e-c99456a3e1dc None admin] RBAC: Authorizing `identity:get_group(group_id=5877fa6c407248d4b6f9fc1a15676f44)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:43.935912 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33131990-23cf-4db3-942e-c99456a3e1dc None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:43.937458 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 222/454] 10.222.0.45 () {56 vars in 1180 bytes} [Wed Aug 24 17:27:43 2022] GET /identity/v3/groups/5877fa6c407248d4b6f9fc1a15676f44 => generated 229 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:27:43.943094 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf94fc70-d66a-4404-9ace-312ff8ce2fd7 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:44.037125 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf94fc70-d66a-4404-9ace-312ff8ce2fd7 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:44.044175 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:44.069456 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f10a1d0-4485-41b2-9271-d1cd8c9053ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:44.070073 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f10a1d0-4485-41b2-9271-d1cd8c9053ed None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:44.070348 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f10a1d0-4485-41b2-9271-d1cd8c9053ed None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:44.070616 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f10a1d0-4485-41b2-9271-d1cd8c9053ed None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:44.071323 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f10a1d0-4485-41b2-9271-d1cd8c9053ed None admin] RBAC: Adding query filter params (group.id=5877fa6c407248d4b6f9fc1a15676f44, role.id=e411ad9eb96e4e8d85b59b5b700ada7b, scope.project.id=1e847003e2ef47df81d36ac4caad47f0) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:27:44.071623 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f10a1d0-4485-41b2-9271-d1cd8c9053ed None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:44.073076 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f10a1d0-4485-41b2-9271-d1cd8c9053ed None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:44.078509 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 233/455] 10.222.0.45 () {56 vars in 1376 bytes} [Wed Aug 24 17:27:43 2022] GET /identity/v3/role_assignments?group.id=5877fa6c407248d4b6f9fc1a15676f44&scope.project.id=1e847003e2ef47df81d36ac4caad47f0&role.id=e411ad9eb96e4e8d85b59b5b700ada7b => generated 611 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:27:46.492350 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee50e593-817e-4a5f-ba36-5886aeef3588 None None] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:46.492894 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee50e593-817e-4a5f-ba36-5886aeef3588 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:46.492894 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee50e593-817e-4a5f-ba36-5886aeef3588 None None] PATH_INFO: `/` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:46.495019 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 223/456] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:27:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:27:46.500421 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e4e7c9-8b6d-459e-9481-d577ae2b9435 None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:46.500855 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e4e7c9-8b6d-459e-9481-d577ae2b9435 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:46.501036 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e4e7c9-8b6d-459e-9481-d577ae2b9435 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:46.528579 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-a5e4e7c9-8b6d-459e-9481-d577ae2b9435 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:27:46.529779 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-a5e4e7c9-8b6d-459e-9481-d577ae2b9435 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:27:46.554759 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adf4975-37d1-4235-b107-14e58d1a4f65 None None] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:46.555034 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adf4975-37d1-4235-b107-14e58d1a4f65 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:46.555354 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adf4975-37d1-4235-b107-14e58d1a4f65 None None] PATH_INFO: `/` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:46.556884 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 224/457] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:27:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:27:46.563626 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a89e8b-201d-4b6a-b025-638d464cc199 None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:46.563906 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a89e8b-201d-4b6a-b025-638d464cc199 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:46.564179 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a89e8b-201d-4b6a-b025-638d464cc199 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:46.588340 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-f6a89e8b-201d-4b6a-b025-638d464cc199 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:27:46.589573 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-f6a89e8b-201d-4b6a-b025-638d464cc199 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:27:46.642164 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 234/458] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:27:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 17:27:46.656387 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:46.692894 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 225/459] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:27:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 17:27:46.875778 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25907d3a-e8ad-4d0d-8179-4960cf151fba None None] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:46.876130 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25907d3a-e8ad-4d0d-8179-4960cf151fba None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:46.876341 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25907d3a-e8ad-4d0d-8179-4960cf151fba None None] PATH_INFO: `/` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:46.877968 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 235/460] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:27:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:27:46.883165 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b3ddd4-7428-4fe2-8410-b7829b052d65 None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:46.883457 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b3ddd4-7428-4fe2-8410-b7829b052d65 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:46.883732 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b3ddd4-7428-4fe2-8410-b7829b052d65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:46.903954 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-06b3ddd4-7428-4fe2-8410-b7829b052d65 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:27:46.909377 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-06b3ddd4-7428-4fe2-8410-b7829b052d65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:27:46.983882 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 226/461] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:27:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:27:47.005979 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25907d3a-e8ad-4d0d-8179-4960cf151fba None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:47.118125 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:47.126090 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25907d3a-e8ad-4d0d-8179-4960cf151fba None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:47.138487 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b3ddd4-7428-4fe2-8410-b7829b052d65 None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:47.180572 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec5ef03-e95e-4f9a-9f99-30847025108e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:47.181300 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec5ef03-e95e-4f9a-9f99-30847025108e None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:47.181577 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec5ef03-e95e-4f9a-9f99-30847025108e None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:47.181847 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec5ef03-e95e-4f9a-9f99-30847025108e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:47.200960 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec5ef03-e95e-4f9a-9f99-30847025108e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:47.202637 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec5ef03-e95e-4f9a-9f99-30847025108e None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:47.226071 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application [None req-2ec5ef03-e95e-4f9a-9f99-30847025108e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:27:47.226071 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:27:47.226071 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:27:47.226071 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:27:47.226071 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:27:47.226071 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:27:47.226071 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:27:47.226071 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:27:47.226071 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:27:47.226071 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:27:47.226071 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:27:47.226071 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:27:47.226071 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:27:47.226071 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:27:47.226071 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:27:47.226071 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:27:47.227012 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:27:47.227012 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:27:47.227012 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:27:47.227012 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:27:47.227012 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:27:47.227012 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:27:47.227012 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:27:47.227012 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application with Lock( Aug 24 17:27:47.227012 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:27:47.227012 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:27:47.227012 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:27:47.227012 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:27:47.227012 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:27:47.227012 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:27:47.227012 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:27:47.227012 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:27:47.227012 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:27:47.227877 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:27:47.227877 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:27:47.227877 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Aug 24 17:27:47.227877 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 236/462] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 17:27:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:27:47.235186 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec5ef03-e95e-4f9a-9f99-30847025108e None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:47.238282 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:47.241342 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b3ddd4-7428-4fe2-8410-b7829b052d65 None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:47.277645 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11234431-8fdc-48e5-aeef-bae8f7780f31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:47.278291 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11234431-8fdc-48e5-aeef-bae8f7780f31 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:47.278575 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11234431-8fdc-48e5-aeef-bae8f7780f31 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:47.278852 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11234431-8fdc-48e5-aeef-bae8f7780f31 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:47.289089 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11234431-8fdc-48e5-aeef-bae8f7780f31 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:47.290537 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11234431-8fdc-48e5-aeef-bae8f7780f31 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:47.306578 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application [None req-11234431-8fdc-48e5-aeef-bae8f7780f31 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:27:47.306578 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:27:47.306578 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:27:47.306578 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:27:47.306578 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:27:47.306578 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:27:47.306578 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:27:47.306578 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:27:47.306578 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:27:47.306578 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:27:47.306578 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:27:47.306578 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:27:47.306578 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:27:47.306578 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:27:47.306578 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:27:47.306578 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:27:47.308070 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:27:47.308070 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:27:47.308070 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:27:47.308070 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:27:47.308070 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:27:47.308070 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:27:47.308070 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:27:47.308070 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application with Lock( Aug 24 17:27:47.308070 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:27:47.308070 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:27:47.308070 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:27:47.308070 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:27:47.308070 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:27:47.308070 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:27:47.308070 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:27:47.308070 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:27:47.308070 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:27:47.308940 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:27:47.308940 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:27:47.308940 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Aug 24 17:27:47.308940 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 227/463] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 17:27:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:27:47.314074 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487e3768-e31b-4201-9991-2c9682a09d11 None None] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:47.314367 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487e3768-e31b-4201-9991-2c9682a09d11 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:47.314646 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487e3768-e31b-4201-9991-2c9682a09d11 None None] PATH_INFO: `/` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:47.316198 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 228/464] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:27:47 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:27:47.317248 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487e3768-e31b-4201-9991-2c9682a09d11 None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:47.361686 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec5ef03-e95e-4f9a-9f99-30847025108e None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:47.397711 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c21fec3-258e-4f2f-9542-60a20e38795e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:47.398321 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c21fec3-258e-4f2f-9542-60a20e38795e None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:47.398874 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c21fec3-258e-4f2f-9542-60a20e38795e None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:47.398874 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c21fec3-258e-4f2f-9542-60a20e38795e None admin] PATH_INFO: `/v3/domains` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:47.399510 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c21fec3-258e-4f2f-9542-60a20e38795e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:27:47.399814 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c21fec3-258e-4f2f-9542-60a20e38795e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:47.401820 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c21fec3-258e-4f2f-9542-60a20e38795e None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:47.414237 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 237/465] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 24 17:27:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:27:47.415059 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c21fec3-258e-4f2f-9542-60a20e38795e None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:47.428440 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487e3768-e31b-4201-9991-2c9682a09d11 None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:47.430451 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:47.463868 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf2d61b-e8ba-44ee-a5ab-85d533ec7719 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:47.464484 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf2d61b-e8ba-44ee-a5ab-85d533ec7719 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:47.464786 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf2d61b-e8ba-44ee-a5ab-85d533ec7719 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:47.465058 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf2d61b-e8ba-44ee-a5ab-85d533ec7719 None admin] PATH_INFO: `/v3/domains` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:47.465693 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf2d61b-e8ba-44ee-a5ab-85d533ec7719 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:27:47.465994 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf2d61b-e8ba-44ee-a5ab-85d533ec7719 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:47.467365 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf2d61b-e8ba-44ee-a5ab-85d533ec7719 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:47.484828 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 229/466] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 24 17:27:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:27:47.486042 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e4b103-1d88-4c81-b603-74503c18be19 None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:47.486364 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e4b103-1d88-4c81-b603-74503c18be19 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:47.486633 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e4b103-1d88-4c81-b603-74503c18be19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:47.504301 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-84e4b103-1d88-4c81-b603-74503c18be19 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:27:47.505496 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-84e4b103-1d88-4c81-b603-74503c18be19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:27:47.514231 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:47.530188 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c21fec3-258e-4f2f-9542-60a20e38795e None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:47.567076 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944befe7-7b07-4227-ac83-7ef4d57120dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:47.567705 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944befe7-7b07-4227-ac83-7ef4d57120dd None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:47.567979 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944befe7-7b07-4227-ac83-7ef4d57120dd None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:47.568698 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944befe7-7b07-4227-ac83-7ef4d57120dd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:47.581877 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944befe7-7b07-4227-ac83-7ef4d57120dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:47.589040 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 230/467] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:27:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:27:47.589040 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e4b103-1d88-4c81-b603-74503c18be19 None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:47.589251 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944befe7-7b07-4227-ac83-7ef4d57120dd None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:47.605365 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application [None req-944befe7-7b07-4227-ac83-7ef4d57120dd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:27:47.605365 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:27:47.605365 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:27:47.605365 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:27:47.605365 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:27:47.605365 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:27:47.605365 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:27:47.605365 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:27:47.605365 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:27:47.605365 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:27:47.605365 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:27:47.605365 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:27:47.605365 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:27:47.605365 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:27:47.605365 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:27:47.605365 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:27:47.606287 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:27:47.606287 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:27:47.606287 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:27:47.606287 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:27:47.606287 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:27:47.606287 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:27:47.606287 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:27:47.606287 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application with Lock( Aug 24 17:27:47.606287 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:27:47.606287 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:27:47.606287 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:27:47.606287 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:27:47.606287 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:27:47.606287 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:27:47.606287 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:27:47.606287 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:27:47.606287 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:27:47.607116 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:27:47.607116 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:27:47.607116 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Aug 24 17:27:47.607116 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 238/468] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 17:27:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:27:47.607331 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944befe7-7b07-4227-ac83-7ef4d57120dd None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:47.693969 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:47.705264 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e4b103-1d88-4c81-b603-74503c18be19 None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:47.742539 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f68e1b-2599-4db4-8a98-fcc1d5c0ff97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:47.743248 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f68e1b-2599-4db4-8a98-fcc1d5c0ff97 None admin] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:47.743535 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f68e1b-2599-4db4-8a98-fcc1d5c0ff97 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:47.743809 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f68e1b-2599-4db4-8a98-fcc1d5c0ff97 None admin] PATH_INFO: `/v3/users` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:47.745041 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f68e1b-2599-4db4-8a98-fcc1d5c0ff97 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:47.746738 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f68e1b-2599-4db4-8a98-fcc1d5c0ff97 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:47.748961 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944befe7-7b07-4227-ac83-7ef4d57120dd None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:47.766682 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:47.776313 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 231/469] 10.222.0.45 () {60 vars in 1132 bytes} [Wed Aug 24 17:27:47 2022] POST /identity/v3/users => generated 246 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:27:47.777440 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f68e1b-2599-4db4-8a98-fcc1d5c0ff97 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:47.821585 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe7798c-5441-424b-9c9a-b6bcca661633 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:47.825291 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe7798c-5441-424b-9c9a-b6bcca661633 None admin] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:47.825814 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe7798c-5441-424b-9c9a-b6bcca661633 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:47.826268 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe7798c-5441-424b-9c9a-b6bcca661633 None admin] PATH_INFO: `/v3/users` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:47.827143 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe7798c-5441-424b-9c9a-b6bcca661633 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:47.831243 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe7798c-5441-424b-9c9a-b6bcca661633 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:47.877651 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 239/470] 10.222.0.45 () {60 vars in 1132 bytes} [Wed Aug 24 17:27:47 2022] POST /identity/v3/users => generated 248 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:27:47.898192 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe7798c-5441-424b-9c9a-b6bcca661633 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:47.966697 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f68e1b-2599-4db4-8a98-fcc1d5c0ff97 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:48.010835 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6dafc4-22cf-43bd-af30-223c62c6128e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:48.011519 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6dafc4-22cf-43bd-af30-223c62c6128e None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:48.011800 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6dafc4-22cf-43bd-af30-223c62c6128e None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:48.012070 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6dafc4-22cf-43bd-af30-223c62c6128e None admin] PATH_INFO: `/v3/domains` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:48.012857 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6dafc4-22cf-43bd-af30-223c62c6128e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:27:48.013152 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6dafc4-22cf-43bd-af30-223c62c6128e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:48.014555 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6dafc4-22cf-43bd-af30-223c62c6128e None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:48.028895 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 232/471] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 24 17:27:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:27:48.030199 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d787ec-24b8-4d5a-9ec4-fd9eaa3f8ebc None None] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:48.030482 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d787ec-24b8-4d5a-9ec4-fd9eaa3f8ebc None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:48.030753 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d787ec-24b8-4d5a-9ec4-fd9eaa3f8ebc None None] PATH_INFO: `/` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:48.035556 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 233/472] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:27:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:27:48.038601 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d787ec-24b8-4d5a-9ec4-fd9eaa3f8ebc None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:48.051633 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe7798c-5441-424b-9c9a-b6bcca661633 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:48.076072 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:48.094907 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a3f458-8a2c-4fd8-b43a-d36eb17b4b63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:48.095638 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a3f458-8a2c-4fd8-b43a-d36eb17b4b63 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:48.095925 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a3f458-8a2c-4fd8-b43a-d36eb17b4b63 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:48.096436 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a3f458-8a2c-4fd8-b43a-d36eb17b4b63 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:48.111707 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a3f458-8a2c-4fd8-b43a-d36eb17b4b63 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:48.113228 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a3f458-8a2c-4fd8-b43a-d36eb17b4b63 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:48.117537 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:48.129841 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application [None req-23a3f458-8a2c-4fd8-b43a-d36eb17b4b63 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:27:48.129841 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:27:48.129841 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:27:48.129841 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:27:48.129841 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:27:48.129841 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:27:48.129841 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:27:48.129841 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:27:48.129841 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:27:48.129841 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:27:48.129841 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:27:48.129841 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:27:48.129841 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:27:48.129841 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:27:48.129841 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:27:48.129841 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:27:48.130786 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:27:48.130786 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:27:48.130786 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:27:48.130786 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:27:48.130786 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:27:48.130786 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:27:48.130786 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:27:48.130786 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application with Lock( Aug 24 17:27:48.130786 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:27:48.130786 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:27:48.130786 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:27:48.130786 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:27:48.130786 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:27:48.130786 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:27:48.130786 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:27:48.130786 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:27:48.130786 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:27:48.131604 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:27:48.131604 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:27:48.131604 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Aug 24 17:27:48.133380 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 240/473] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 17:27:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 17:27:48.135136 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cca5e3-b4cc-4beb-afaa-2caa073268dc None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:48.135623 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cca5e3-b4cc-4beb-afaa-2caa073268dc None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:48.136086 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cca5e3-b4cc-4beb-afaa-2caa073268dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:48.155201 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-83cca5e3-b4cc-4beb-afaa-2caa073268dc None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:27:48.156743 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-83cca5e3-b4cc-4beb-afaa-2caa073268dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:27:48.175028 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d787ec-24b8-4d5a-9ec4-fd9eaa3f8ebc None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:48.218950 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8b4132-5186-49a8-aaa9-e7d070c13cb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:48.220039 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8b4132-5186-49a8-aaa9-e7d070c13cb7 None admin] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:48.220630 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8b4132-5186-49a8-aaa9-e7d070c13cb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:48.221113 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8b4132-5186-49a8-aaa9-e7d070c13cb7 None admin] PATH_INFO: `/v3/users` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:48.221983 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8b4132-5186-49a8-aaa9-e7d070c13cb7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:48.223724 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8b4132-5186-49a8-aaa9-e7d070c13cb7 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:48.236120 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 241/474] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:27:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:27:48.237368 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cca5e3-b4cc-4beb-afaa-2caa073268dc None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:48.251747 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 234/475] 10.222.0.45 () {60 vars in 1132 bytes} [Wed Aug 24 17:27:48 2022] POST /identity/v3/users => generated 248 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:27:48.361417 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cca5e3-b4cc-4beb-afaa-2caa073268dc None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:48.363275 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:48.419431 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e86abe-4cde-4317-99b1-f1e0fea9d0dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:48.420448 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e86abe-4cde-4317-99b1-f1e0fea9d0dc None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:48.420962 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e86abe-4cde-4317-99b1-f1e0fea9d0dc None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:48.421463 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e86abe-4cde-4317-99b1-f1e0fea9d0dc None admin] PATH_INFO: `/v3/domains` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:48.422317 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e86abe-4cde-4317-99b1-f1e0fea9d0dc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:27:48.422814 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e86abe-4cde-4317-99b1-f1e0fea9d0dc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:48.424472 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e86abe-4cde-4317-99b1-f1e0fea9d0dc None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:48.446590 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 242/476] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 24 17:27:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:27:48.452078 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8b4132-5186-49a8-aaa9-e7d070c13cb7 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:48.573639 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e86abe-4cde-4317-99b1-f1e0fea9d0dc None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:48.584389 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8b4132-5186-49a8-aaa9-e7d070c13cb7 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:48.599993 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:48.621053 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf678196-804a-4cd0-b2e7-3f97d98d7040 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:48.621668 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf678196-804a-4cd0-b2e7-3f97d98d7040 None admin] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:48.621944 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf678196-804a-4cd0-b2e7-3f97d98d7040 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:48.622214 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf678196-804a-4cd0-b2e7-3f97d98d7040 None admin] PATH_INFO: `/v3/users` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:48.623700 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf678196-804a-4cd0-b2e7-3f97d98d7040 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:48.624303 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf678196-804a-4cd0-b2e7-3f97d98d7040 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:48.649703 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 235/477] 10.222.0.45 () {60 vars in 1132 bytes} [Wed Aug 24 17:27:48 2022] POST /identity/v3/users => generated 249 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:27:48.701669 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e86abe-4cde-4317-99b1-f1e0fea9d0dc None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:48.742341 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3342c1f9-4b1d-4ba0-b8e0-9ed76d8d22c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:48.743334 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3342c1f9-4b1d-4ba0-b8e0-9ed76d8d22c1 None admin] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:48.743812 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3342c1f9-4b1d-4ba0-b8e0-9ed76d8d22c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:48.744315 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3342c1f9-4b1d-4ba0-b8e0-9ed76d8d22c1 None admin] PATH_INFO: `/v3/roles` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:48.745752 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3342c1f9-4b1d-4ba0-b8e0-9ed76d8d22c1 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:48.747456 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3342c1f9-4b1d-4ba0-b8e0-9ed76d8d22c1 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:48.757422 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.sql.core [None req-3342c1f9-4b1d-4ba0-b8e0-9ed76d8d22c1 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 24 17:27:48.757422 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 24 17:27:48.757422 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [parameters: {'id': '09801f3160954bf7a574791fbe923dc9', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 24 17:27:48.757422 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81758) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 24 17:27:48.776927 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application [None req-3342c1f9-4b1d-4ba0-b8e0-9ed76d8d22c1 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 24 17:27:48.776927 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:27:48.776927 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 17:27:48.776927 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 17:27:48.776927 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 17:27:48.776927 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 17:27:48.776927 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 17:27:48.776927 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 17:27:48.776927 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 17:27:48.776927 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application conn.query(q) Aug 24 17:27:48.776927 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 17:27:48.776927 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 17:27:48.776927 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 17:27:48.776927 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application result.read() Aug 24 17:27:48.776927 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 17:27:48.777876 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 17:27:48.777876 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 17:27:48.777876 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 17:27:48.777876 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 17:27:48.777876 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 17:27:48.777876 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 17:27:48.777876 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 17:27:48.777876 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 24 17:27:48.777876 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Aug 24 17:27:48.777876 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 24 17:27:48.777876 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Aug 24 17:27:48.777876 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:27:48.777876 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 24 17:27:48.777876 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 24 17:27:48.777876 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Aug 24 17:27:48.777876 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return ref.to_dict() Aug 24 17:27:48.777876 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 17:27:48.777876 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application next(self.gen) Aug 24 17:27:48.778808 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 24 17:27:48.778808 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application yield resource Aug 24 17:27:48.778808 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 17:27:48.778808 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application next(self.gen) Aug 24 17:27:48.778808 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 24 17:27:48.778808 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application self.session.rollback() Aug 24 17:27:48.778808 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 24 17:27:48.778808 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application self.force_reraise() Aug 24 17:27:48.778808 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 24 17:27:48.778808 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application raise self.value Aug 24 17:27:48.778808 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 24 17:27:48.778808 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 24 17:27:48.778808 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 24 17:27:48.778808 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application session.commit() Aug 24 17:27:48.778808 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 24 17:27:48.778808 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 24 17:27:48.778808 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 24 17:27:48.779668 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application self._prepare_impl() Aug 24 17:27:48.779668 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 24 17:27:48.779668 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application self.session.flush() Aug 24 17:27:48.779668 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 24 17:27:48.779668 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application self._flush(objects) Aug 24 17:27:48.779668 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 24 17:27:48.779668 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 24 17:27:48.779668 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 24 17:27:48.779668 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application compat.raise_( Aug 24 17:27:48.779668 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 17:27:48.779668 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application raise exception Aug 24 17:27:48.779668 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 24 17:27:48.779668 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application flush_context.execute() Aug 24 17:27:48.779668 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 24 17:27:48.779668 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application rec.execute(self) Aug 24 17:27:48.779668 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 24 17:27:48.779668 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 24 17:27:48.780577 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 24 17:27:48.780577 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 24 17:27:48.780577 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 24 17:27:48.780577 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 24 17:27:48.780577 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 24 17:27:48.780577 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 24 17:27:48.780577 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 24 17:27:48.780577 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 24 17:27:48.780577 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 24 17:27:48.780577 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 24 17:27:48.780577 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 24 17:27:48.780577 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 24 17:27:48.780577 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 24 17:27:48.780577 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 24 17:27:48.780577 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 17:27:48.780577 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application raise exception Aug 24 17:27:48.781384 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 17:27:48.781384 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 17:27:48.781384 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 17:27:48.781384 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 17:27:48.781384 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 17:27:48.781384 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 17:27:48.781384 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 17:27:48.781384 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application conn.query(q) Aug 24 17:27:48.781384 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 17:27:48.781384 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 17:27:48.781384 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 17:27:48.781384 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application result.read() Aug 24 17:27:48.781384 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 17:27:48.781384 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 17:27:48.781384 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 17:27:48.781384 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 17:27:48.782142 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 17:27:48.782142 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 17:27:48.782142 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 17:27:48.782142 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 17:27:48.782142 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 24 17:27:48.782142 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 24 17:27:48.782142 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application [parameters: {'id': '09801f3160954bf7a574791fbe923dc9', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 24 17:27:48.782142 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 24 17:27:48.782142 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Aug 24 17:27:48.782142 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 17:27:48.782142 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Aug 24 17:27:48.782142 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:27:48.782142 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:27:48.782142 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:27:48.782142 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:27:48.782142 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:27:48.782927 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:27:48.782927 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:27:48.782927 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:27:48.782927 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:27:48.782927 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:27:48.782927 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:27:48.782927 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Aug 24 17:27:48.782927 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Aug 24 17:27:48.782927 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:27:48.782927 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:27:48.782927 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Aug 24 17:27:48.782927 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Aug 24 17:27:48.782927 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 24 17:27:48.782927 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 24 17:27:48.782927 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 24 17:27:48.782927 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Aug 24 17:27:48.786984 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 243/478] 10.222.0.45 () {60 vars in 1131 bytes} [Wed Aug 24 17:27:48 2022] POST /identity/v3/roles => generated 144 bytes in 214 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:27:48.795632 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf678196-804a-4cd0-b2e7-3f97d98d7040 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:48.913070 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf678196-804a-4cd0-b2e7-3f97d98d7040 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:48.949179 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16979577-3719-4610-b533-006998e7b6a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:48.950209 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16979577-3719-4610-b533-006998e7b6a8 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:48.950737 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16979577-3719-4610-b533-006998e7b6a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:48.951233 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16979577-3719-4610-b533-006998e7b6a8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:48.962289 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16979577-3719-4610-b533-006998e7b6a8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:27:48.968662 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application [None req-16979577-3719-4610-b533-006998e7b6a8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 17:27:48.968662 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:27:48.968662 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:27:48.968662 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:27:48.968662 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:27:48.968662 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:27:48.968662 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:27:48.968662 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:27:48.968662 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:27:48.968662 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:27:48.968662 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:27:48.968662 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:27:48.968662 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:27:48.968662 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:27:48.968662 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:27:48.968662 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:27:48.969532 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:27:48.969532 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:27:48.969532 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:27:48.969532 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:27:48.969532 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:27:48.969532 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:27:48.969532 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:27:48.969532 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:27:48.969532 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:27:48.969532 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:27:48.969532 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:27:48.969532 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application with Lock( Aug 24 17:27:48.969532 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:27:48.969532 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:27:48.969532 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:27:48.969532 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:27:48.970303 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:27:48.970303 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:27:48.970303 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:27:48.970303 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:27:48.970303 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:27:48.970303 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:27:48.970303 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:27:48.970303 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:27:48.970303 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:27:48.970303 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:27:48.970303 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 17:27:48.970303 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Aug 24 17:27:48.972884 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 236/479] 10.222.0.45 () {56 vars in 1114 bytes} [Wed Aug 24 17:27:48 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:27:48.978732 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3342c1f9-4b1d-4ba0-b8e0-9ed76d8d22c1 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:49.011163 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:49.092633 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3342c1f9-4b1d-4ba0-b8e0-9ed76d8d22c1 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:49.132007 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896662fd-3189-4c59-b17f-ebb82701ec10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:49.132685 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896662fd-3189-4c59-b17f-ebb82701ec10 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:49.132973 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896662fd-3189-4c59-b17f-ebb82701ec10 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:49.133247 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896662fd-3189-4c59-b17f-ebb82701ec10 None admin] PATH_INFO: `/v3/roles` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:49.133914 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896662fd-3189-4c59-b17f-ebb82701ec10 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:27:49.134217 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896662fd-3189-4c59-b17f-ebb82701ec10 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:49.135620 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896662fd-3189-4c59-b17f-ebb82701ec10 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:49.144192 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 244/480] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Aug 24 17:27:48 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:27:50.064038 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e0d514-526d-4fa4-afff-a131590b8e79 None None] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:50.064525 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e0d514-526d-4fa4-afff-a131590b8e79 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:50.064525 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e0d514-526d-4fa4-afff-a131590b8e79 None None] PATH_INFO: `/` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:50.066061 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 237/481] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:27:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:27:50.071537 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dbef85-6fdb-49fa-9495-19474eee2c84 None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:50.071966 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dbef85-6fdb-49fa-9495-19474eee2c84 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:50.072062 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dbef85-6fdb-49fa-9495-19474eee2c84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:50.097489 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-e5dbef85-6fdb-49fa-9495-19474eee2c84 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:27:50.098676 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-e5dbef85-6fdb-49fa-9495-19474eee2c84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:27:50.180588 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852824a9-a7a5-4019-b4a7-3534ebd4ed0b None None] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:50.181002 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852824a9-a7a5-4019-b4a7-3534ebd4ed0b None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:50.181351 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852824a9-a7a5-4019-b4a7-3534ebd4ed0b None None] PATH_INFO: `/` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:50.182533 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 245/482] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:27:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 17:27:50.183587 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 238/483] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:27:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:27:50.188915 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d637876-b8dd-433b-8a70-bdfc02bff5e6 None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:50.189318 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d637876-b8dd-433b-8a70-bdfc02bff5e6 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:50.189597 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d637876-b8dd-433b-8a70-bdfc02bff5e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:50.218476 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-9d637876-b8dd-433b-8a70-bdfc02bff5e6 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:27:50.219652 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-9d637876-b8dd-433b-8a70-bdfc02bff5e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:27:50.240584 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:50.302761 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 246/484] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:27:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:27:50.568059 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852824a9-a7a5-4019-b4a7-3534ebd4ed0b None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:50.605108 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:50.694801 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852824a9-a7a5-4019-b4a7-3534ebd4ed0b None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:50.738829 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b2c510-251f-452d-8bcb-78ad7f32a86d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:50.739471 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b2c510-251f-452d-8bcb-78ad7f32a86d None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:50.739747 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b2c510-251f-452d-8bcb-78ad7f32a86d None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:50.740015 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b2c510-251f-452d-8bcb-78ad7f32a86d None admin] PATH_INFO: `/v3/roles/service` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:50.757384 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b2c510-251f-452d-8bcb-78ad7f32a86d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:27:50.762142 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application [None req-00b2c510-251f-452d-8bcb-78ad7f32a86d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 17:27:50.762142 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:27:50.762142 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:27:50.762142 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:27:50.762142 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:27:50.762142 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:27:50.762142 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:27:50.762142 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:27:50.762142 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:27:50.762142 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:27:50.762142 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:27:50.762142 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:27:50.762142 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:27:50.762142 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:27:50.762142 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:27:50.762142 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:27:50.762993 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:27:50.762993 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:27:50.762993 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:27:50.762993 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:27:50.762993 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:27:50.762993 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:27:50.762993 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:27:50.762993 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:27:50.762993 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:27:50.762993 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:27:50.762993 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:27:50.762993 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application with Lock( Aug 24 17:27:50.762993 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:27:50.762993 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:27:50.762993 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:27:50.762993 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:27:50.763767 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:27:50.763767 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:27:50.763767 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:27:50.763767 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:27:50.763767 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:27:50.763767 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:27:50.763767 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:27:50.763767 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:27:50.763767 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:27:50.763767 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:27:50.763767 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 17:27:50.763767 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Aug 24 17:27:50.763767 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 239/485] 10.222.0.45 () {56 vars in 1102 bytes} [Wed Aug 24 17:27:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:27:50.770679 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d637876-b8dd-433b-8a70-bdfc02bff5e6 None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:50.813070 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b2c510-251f-452d-8bcb-78ad7f32a86d None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:50.917247 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d637876-b8dd-433b-8a70-bdfc02bff5e6 None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:50.917481 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:50.934924 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b2c510-251f-452d-8bcb-78ad7f32a86d None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:50.954427 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c101b99-d380-423e-bab8-9fe7d1a7b4c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:50.955181 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c101b99-d380-423e-bab8-9fe7d1a7b4c2 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:50.955561 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c101b99-d380-423e-bab8-9fe7d1a7b4c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:50.956016 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c101b99-d380-423e-bab8-9fe7d1a7b4c2 None admin] PATH_INFO: `/v3/roles` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:50.956723 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c101b99-d380-423e-bab8-9fe7d1a7b4c2 None admin] RBAC: Adding query filter params (name=service) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:27:50.957127 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c101b99-d380-423e-bab8-9fe7d1a7b4c2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:50.958600 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c101b99-d380-423e-bab8-9fe7d1a7b4c2 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:50.967416 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 247/486] 10.222.0.45 () {56 vars in 1103 bytes} [Wed Aug 24 17:27:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:27:50.973281 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d592d42b-485b-4263-89cf-e69dfd9cd5b1 None None] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:50.973564 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d592d42b-485b-4263-89cf-e69dfd9cd5b1 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:50.973835 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d592d42b-485b-4263-89cf-e69dfd9cd5b1 None None] PATH_INFO: `/` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:50.975221 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 248/487] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:27:50 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:27:50.976776 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d592d42b-485b-4263-89cf-e69dfd9cd5b1 None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:50.981215 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5114ccd-afc2-4c8e-b7a0-c4adaa6eb61a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:50.981848 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5114ccd-afc2-4c8e-b7a0-c4adaa6eb61a None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:50.982122 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5114ccd-afc2-4c8e-b7a0-c4adaa6eb61a None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:50.982390 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5114ccd-afc2-4c8e-b7a0-c4adaa6eb61a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:50.994628 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5114ccd-afc2-4c8e-b7a0-c4adaa6eb61a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:27:50.995507 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:51.000740 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application [None req-d5114ccd-afc2-4c8e-b7a0-c4adaa6eb61a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 17:27:51.000740 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:27:51.000740 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:27:51.000740 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:27:51.000740 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:27:51.000740 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:27:51.000740 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:27:51.000740 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:27:51.000740 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:27:51.000740 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:27:51.000740 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:27:51.000740 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:27:51.000740 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:27:51.000740 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:27:51.000740 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:27:51.000740 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:27:51.001626 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:27:51.001626 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:27:51.001626 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:27:51.001626 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:27:51.001626 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:27:51.001626 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:27:51.001626 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:27:51.001626 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:27:51.001626 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:27:51.001626 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:27:51.001626 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:27:51.001626 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application with Lock( Aug 24 17:27:51.001626 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:27:51.001626 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:27:51.001626 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:27:51.001626 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:27:51.002392 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:27:51.002392 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:27:51.002392 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:27:51.002392 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:27:51.002392 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:27:51.002392 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:27:51.002392 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:27:51.002392 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:27:51.002392 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:27:51.002392 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:27:51.002392 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 17:27:51.002392 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Aug 24 17:27:51.002392 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 240/488] 10.222.0.45 () {56 vars in 1102 bytes} [Wed Aug 24 17:27:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 17:27:51.007862 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430a9c42-6e7d-4cd4-ae0c-de05e725eca4 None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:51.008145 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430a9c42-6e7d-4cd4-ae0c-de05e725eca4 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:51.008415 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430a9c42-6e7d-4cd4-ae0c-de05e725eca4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:51.028070 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-430a9c42-6e7d-4cd4-ae0c-de05e725eca4 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:27:51.029218 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-430a9c42-6e7d-4cd4-ae0c-de05e725eca4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:27:51.098480 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d592d42b-485b-4263-89cf-e69dfd9cd5b1 None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:51.099009 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 241/489] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:27:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:27:51.104707 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430a9c42-6e7d-4cd4-ae0c-de05e725eca4 None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:51.173372 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d3aa46-13d2-4348-b85d-0ec05f9e5023 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:51.173949 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d3aa46-13d2-4348-b85d-0ec05f9e5023 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:51.174236 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d3aa46-13d2-4348-b85d-0ec05f9e5023 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:51.174517 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d3aa46-13d2-4348-b85d-0ec05f9e5023 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:51.185172 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:51.205635 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d3aa46-13d2-4348-b85d-0ec05f9e5023 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:51.207133 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d3aa46-13d2-4348-b85d-0ec05f9e5023 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:51.207358 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430a9c42-6e7d-4cd4-ae0c-de05e725eca4 None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:51.238669 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application [None req-27d3aa46-13d2-4348-b85d-0ec05f9e5023 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:27:51.238669 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:27:51.238669 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:27:51.238669 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:27:51.238669 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:27:51.238669 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:27:51.238669 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:27:51.238669 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:27:51.238669 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:27:51.238669 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:27:51.238669 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:27:51.238669 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:27:51.238669 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:27:51.238669 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:27:51.238669 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:27:51.238669 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:27:51.239691 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:27:51.239691 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:27:51.239691 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:27:51.239691 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:27:51.239691 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:27:51.239691 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:27:51.239691 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:27:51.239691 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application with Lock( Aug 24 17:27:51.239691 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:27:51.239691 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:27:51.239691 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:27:51.239691 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:27:51.239691 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:27:51.239691 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:27:51.239691 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:27:51.239691 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:27:51.239691 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:27:51.240519 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:27:51.240519 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:27:51.240519 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Aug 24 17:27:51.240519 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 249/490] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 17:27:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:27:51.244840 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3959e5a-0ff6-4532-bb96-4ddac8fe9450 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:51.245524 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3959e5a-0ff6-4532-bb96-4ddac8fe9450 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:51.245886 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3959e5a-0ff6-4532-bb96-4ddac8fe9450 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:51.246226 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3959e5a-0ff6-4532-bb96-4ddac8fe9450 None admin] PATH_INFO: `/v3/roles` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:51.246971 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3959e5a-0ff6-4532-bb96-4ddac8fe9450 None admin] RBAC: Adding query filter params (name=service) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:27:51.247447 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3959e5a-0ff6-4532-bb96-4ddac8fe9450 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:51.248389 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d3aa46-13d2-4348-b85d-0ec05f9e5023 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:51.248907 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3959e5a-0ff6-4532-bb96-4ddac8fe9450 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:51.258132 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 242/491] 10.222.0.45 () {56 vars in 1103 bytes} [Wed Aug 24 17:27:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:27:51.263868 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3959e5a-0ff6-4532-bb96-4ddac8fe9450 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:51.369286 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3959e5a-0ff6-4532-bb96-4ddac8fe9450 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:51.405202 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ce2d04-384d-487a-8c5b-c8fc3ad8cad9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:51.405828 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ce2d04-384d-487a-8c5b-c8fc3ad8cad9 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:51.406155 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ce2d04-384d-487a-8c5b-c8fc3ad8cad9 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:51.406474 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ce2d04-384d-487a-8c5b-c8fc3ad8cad9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:51.407869 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:51.417895 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ce2d04-384d-487a-8c5b-c8fc3ad8cad9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:51.419370 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ce2d04-384d-487a-8c5b-c8fc3ad8cad9 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:51.421139 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:51.435818 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application [None req-93ce2d04-384d-487a-8c5b-c8fc3ad8cad9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:27:51.435818 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:27:51.435818 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:27:51.435818 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:27:51.435818 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:27:51.435818 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:27:51.435818 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:27:51.435818 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:27:51.435818 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:27:51.435818 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:27:51.435818 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:27:51.435818 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:27:51.435818 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:27:51.435818 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:27:51.435818 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:27:51.435818 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:27:51.436759 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:27:51.436759 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:27:51.436759 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:27:51.436759 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:27:51.436759 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:27:51.436759 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:27:51.436759 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:27:51.436759 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application with Lock( Aug 24 17:27:51.436759 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:27:51.436759 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:27:51.436759 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:27:51.436759 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:27:51.436759 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:27:51.436759 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:27:51.436759 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:27:51.436759 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:27:51.436759 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:27:51.437602 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:27:51.437602 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:27:51.437602 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Aug 24 17:27:51.437602 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 243/492] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 17:27:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 17:27:51.445424 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ce2d04-384d-487a-8c5b-c8fc3ad8cad9 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:51.469541 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d3aa46-13d2-4348-b85d-0ec05f9e5023 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:51.505100 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953ec7db-c289-4e50-adcb-ebf6794c71fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:51.505709 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953ec7db-c289-4e50-adcb-ebf6794c71fe None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:51.505990 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953ec7db-c289-4e50-adcb-ebf6794c71fe None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:51.506263 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953ec7db-c289-4e50-adcb-ebf6794c71fe None admin] PATH_INFO: `/v3/domains` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:51.506907 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953ec7db-c289-4e50-adcb-ebf6794c71fe None admin] RBAC: Adding query filter params (name=Default) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:27:51.507218 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953ec7db-c289-4e50-adcb-ebf6794c71fe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:51.508633 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953ec7db-c289-4e50-adcb-ebf6794c71fe None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:51.521426 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 250/493] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 24 17:27:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:27:51.522561 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293af998-f8ec-47ce-aa09-3bb07f5ba135 None None] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:51.522922 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293af998-f8ec-47ce-aa09-3bb07f5ba135 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:51.523274 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293af998-f8ec-47ce-aa09-3bb07f5ba135 None None] PATH_INFO: `/` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:51.524686 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 251/494] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:27:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:27:51.529422 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293af998-f8ec-47ce-aa09-3bb07f5ba135 None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:51.562743 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ce2d04-384d-487a-8c5b-c8fc3ad8cad9 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:51.598645 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ab211e-4cd9-478e-ab8b-2975fcb12b9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:51.599253 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ab211e-4cd9-478e-ab8b-2975fcb12b9d None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:51.599529 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ab211e-4cd9-478e-ab8b-2975fcb12b9d None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:51.599799 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ab211e-4cd9-478e-ab8b-2975fcb12b9d None admin] PATH_INFO: `/v3/domains` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:51.600452 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ab211e-4cd9-478e-ab8b-2975fcb12b9d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:27:51.600789 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ab211e-4cd9-478e-ab8b-2975fcb12b9d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:51.602158 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ab211e-4cd9-478e-ab8b-2975fcb12b9d None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:51.613858 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 244/495] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 24 17:27:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:27:51.614643 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ab211e-4cd9-478e-ab8b-2975fcb12b9d None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:51.634390 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293af998-f8ec-47ce-aa09-3bb07f5ba135 None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:51.656600 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:51.668634 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7192612-dd70-49bb-a40f-d2ce36b78007 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:51.669262 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7192612-dd70-49bb-a40f-d2ce36b78007 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:51.669537 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7192612-dd70-49bb-a40f-d2ce36b78007 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:51.669805 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7192612-dd70-49bb-a40f-d2ce36b78007 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:51.670856 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:51.680087 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7192612-dd70-49bb-a40f-d2ce36b78007 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:27:51.685283 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application [None req-e7192612-dd70-49bb-a40f-d2ce36b78007 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 17:27:51.685283 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:27:51.685283 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:27:51.685283 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:27:51.685283 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:27:51.685283 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:27:51.685283 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:27:51.685283 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:27:51.685283 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:27:51.685283 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:27:51.685283 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:27:51.685283 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:27:51.685283 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:27:51.685283 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:27:51.685283 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:27:51.685283 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:27:51.686267 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:27:51.686267 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:27:51.686267 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:27:51.686267 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:27:51.686267 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:27:51.686267 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:27:51.686267 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:27:51.686267 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:27:51.686267 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:27:51.686267 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:27:51.686267 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:27:51.686267 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application with Lock( Aug 24 17:27:51.686267 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:27:51.686267 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:27:51.686267 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:27:51.686267 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:27:51.687109 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:27:51.687109 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:27:51.687109 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:27:51.687109 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:27:51.687109 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:27:51.687109 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:27:51.687109 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:27:51.687109 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:27:51.687109 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:27:51.687109 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:27:51.687109 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 17:27:51.687109 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Aug 24 17:27:51.687109 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 252/496] 10.222.0.45 () {56 vars in 1102 bytes} [Wed Aug 24 17:27:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:27:51.687796 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a34027-5c4b-44dd-bbee-e96709d46176 None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:51.687865 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a34027-5c4b-44dd-bbee-e96709d46176 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:51.688091 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a34027-5c4b-44dd-bbee-e96709d46176 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:51.705478 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-64a34027-5c4b-44dd-bbee-e96709d46176 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:27:51.706735 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-64a34027-5c4b-44dd-bbee-e96709d46176 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:27:51.719831 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ab211e-4cd9-478e-ab8b-2975fcb12b9d None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:51.754740 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3e3199-46d3-445b-a484-37def9262105 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:51.755383 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3e3199-46d3-445b-a484-37def9262105 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:51.755637 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3e3199-46d3-445b-a484-37def9262105 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:51.755911 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3e3199-46d3-445b-a484-37def9262105 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:51.764720 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3e3199-46d3-445b-a484-37def9262105 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:51.766208 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3e3199-46d3-445b-a484-37def9262105 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:51.772645 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 253/497] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:27:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:27:51.784199 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27184e8e-0795-4702-a779-f5d00705d3f5 None None] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:51.784199 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27184e8e-0795-4702-a779-f5d00705d3f5 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:51.784199 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27184e8e-0795-4702-a779-f5d00705d3f5 None None] PATH_INFO: `/` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:51.784199 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 254/498] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:27:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:27:51.784199 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27184e8e-0795-4702-a779-f5d00705d3f5 None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:51.790643 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application [None req-6b3e3199-46d3-445b-a484-37def9262105 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 17:27:51.790643 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:27:51.790643 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:27:51.790643 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:27:51.790643 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:27:51.790643 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:27:51.790643 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:27:51.790643 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:27:51.790643 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:27:51.790643 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:27:51.790643 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:27:51.790643 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:27:51.790643 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 17:27:51.790643 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 17:27:51.790643 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 17:27:51.790643 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 17:27:51.792089 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:27:51.792089 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:27:51.792089 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 17:27:51.792089 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:27:51.792089 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 17:27:51.792089 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:27:51.792089 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:27:51.792089 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:27:51.792089 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:27:51.792089 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:27:51.792089 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:27:51.792089 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application with Lock( Aug 24 17:27:51.792089 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:27:51.792089 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:27:51.792089 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:27:51.792089 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:27:51.792089 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:27:51.792089 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:27:51.793002 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:27:51.793002 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:27:51.793002 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 17:27:51.793002 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 17:27:51.793002 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 17:27:51.793002 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 17:27:51.793002 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 17:27:51.793002 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 17:27:51.793002 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 17:27:51.793002 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Aug 24 17:27:51.797214 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 245/499] 10.222.0.45 () {56 vars in 1093 bytes} [Wed Aug 24 17:27:51 2022] GET /identity/v3/users/nova => generated 82 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:27:51.798069 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3e3199-46d3-445b-a484-37def9262105 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:51.891145 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27184e8e-0795-4702-a779-f5d00705d3f5 None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:51.901544 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3e3199-46d3-445b-a484-37def9262105 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:51.904916 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:51.917713 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:51.928144 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d35d5e5-8380-4f63-82f8-f2fe10f4fc75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:51.928796 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d35d5e5-8380-4f63-82f8-f2fe10f4fc75 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:51.929075 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d35d5e5-8380-4f63-82f8-f2fe10f4fc75 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:51.929345 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d35d5e5-8380-4f63-82f8-f2fe10f4fc75 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:51.937683 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d35d5e5-8380-4f63-82f8-f2fe10f4fc75 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:51.939087 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d35d5e5-8380-4f63-82f8-f2fe10f4fc75 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:51.940384 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adca653-b2fd-406d-b36c-2b571ca576dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:51.941023 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adca653-b2fd-406d-b36c-2b571ca576dc None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:51.941291 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adca653-b2fd-406d-b36c-2b571ca576dc None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:51.941889 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adca653-b2fd-406d-b36c-2b571ca576dc None admin] PATH_INFO: `/v3/roles` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:51.942214 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adca653-b2fd-406d-b36c-2b571ca576dc None admin] RBAC: Adding query filter params (name=service) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:27:51.942516 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adca653-b2fd-406d-b36c-2b571ca576dc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:51.943880 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adca653-b2fd-406d-b36c-2b571ca576dc None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:51.952279 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 246/500] 10.222.0.45 () {56 vars in 1103 bytes} [Wed Aug 24 17:27:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:27:51.952923 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application [None req-3d35d5e5-8380-4f63-82f8-f2fe10f4fc75 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 24 17:27:51.952923 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:27:51.952923 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:27:51.952923 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:27:51.952923 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:27:51.952923 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:27:51.952923 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:27:51.952923 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:27:51.952923 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:27:51.952923 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:27:51.952923 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:27:51.952923 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:27:51.952923 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 17:27:51.952923 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 17:27:51.952923 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 17:27:51.952923 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 17:27:51.953753 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:27:51.953753 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:27:51.953753 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 17:27:51.953753 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:27:51.953753 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 17:27:51.953753 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:27:51.953753 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:27:51.953753 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:27:51.953753 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:27:51.953753 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:27:51.953753 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:27:51.953753 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application with Lock( Aug 24 17:27:51.953753 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:27:51.953753 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:27:51.953753 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:27:51.953753 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:27:51.953753 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:27:51.953753 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:27:51.954623 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:27:51.954623 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:27:51.954623 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 17:27:51.954623 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 17:27:51.954623 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 17:27:51.954623 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 17:27:51.954623 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 17:27:51.954623 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 17:27:51.954623 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 24 17:27:51.954623 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Aug 24 17:27:51.955111 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b208883-7037-40b3-824c-f5b7d9188093 None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:51.955111 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b208883-7037-40b3-824c-f5b7d9188093 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:51.955111 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b208883-7037-40b3-824c-f5b7d9188093 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:51.957205 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 255/501] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Aug 24 17:27:51 2022] GET /identity/v3/users/glance => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:27:51.958346 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d35d5e5-8380-4f63-82f8-f2fe10f4fc75 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:51.973845 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-5b208883-7037-40b3-824c-f5b7d9188093 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:27:51.975603 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-5b208883-7037-40b3-824c-f5b7d9188093 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:27:52.046925 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 247/502] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:27:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:27:52.047828 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b208883-7037-40b3-824c-f5b7d9188093 None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:52.066207 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d35d5e5-8380-4f63-82f8-f2fe10f4fc75 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:52.105127 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4939b7-cd67-4a33-a262-c71edbede73f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:52.105737 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4939b7-cd67-4a33-a262-c71edbede73f None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:52.105994 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4939b7-cd67-4a33-a262-c71edbede73f None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:52.106267 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4939b7-cd67-4a33-a262-c71edbede73f None admin] PATH_INFO: `/v3/users` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:52.106990 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4939b7-cd67-4a33-a262-c71edbede73f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:27:52.107295 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4939b7-cd67-4a33-a262-c71edbede73f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:52.108754 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4939b7-cd67-4a33-a262-c71edbede73f None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:52.151585 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 256/503] 10.222.0.45 () {56 vars in 1133 bytes} [Wed Aug 24 17:27:51 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:27:52.151585 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4939b7-cd67-4a33-a262-c71edbede73f None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:52.156517 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b208883-7037-40b3-824c-f5b7d9188093 None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:52.164779 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:52.182508 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:52.197283 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1f7851-c21b-4a41-aae0-45d79415c899 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:52.197874 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1f7851-c21b-4a41-aae0-45d79415c899 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:52.198151 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1f7851-c21b-4a41-aae0-45d79415c899 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:52.198531 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1f7851-c21b-4a41-aae0-45d79415c899 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:52.209376 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1f7851-c21b-4a41-aae0-45d79415c899 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:52.211091 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1f7851-c21b-4a41-aae0-45d79415c899 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:52.227727 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application [None req-8e1f7851-c21b-4a41-aae0-45d79415c899 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:27:52.227727 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:27:52.227727 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:27:52.227727 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:27:52.227727 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:27:52.227727 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:27:52.227727 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:27:52.227727 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:27:52.227727 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:27:52.227727 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:27:52.227727 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:27:52.227727 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:27:52.227727 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:27:52.227727 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:27:52.227727 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:27:52.227727 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:27:52.228689 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:27:52.228689 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:27:52.228689 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:27:52.228689 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:27:52.228689 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:27:52.228689 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:27:52.228689 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:27:52.228689 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application with Lock( Aug 24 17:27:52.228689 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:27:52.228689 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:27:52.228689 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:27:52.228689 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:27:52.228689 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:27:52.228689 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:27:52.228689 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:27:52.228689 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:27:52.228689 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:27:52.230146 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:27:52.230146 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:27:52.230146 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Aug 24 17:27:52.230146 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 248/504] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 17:27:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:27:52.230367 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1f7851-c21b-4a41-aae0-45d79415c899 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:52.258252 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4939b7-cd67-4a33-a262-c71edbede73f None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:52.293076 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534c89d4-e041-4757-ade6-38ee7065e57b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:52.293678 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534c89d4-e041-4757-ade6-38ee7065e57b None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:52.293956 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534c89d4-e041-4757-ade6-38ee7065e57b None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:52.294225 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534c89d4-e041-4757-ade6-38ee7065e57b None admin] PATH_INFO: `/v3/users` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:52.294942 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534c89d4-e041-4757-ade6-38ee7065e57b None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:27:52.295242 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534c89d4-e041-4757-ade6-38ee7065e57b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:52.296691 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534c89d4-e041-4757-ade6-38ee7065e57b None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:52.313830 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 257/505] 10.222.0.45 () {56 vars in 1137 bytes} [Wed Aug 24 17:27:52 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:27:52.314627 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534c89d4-e041-4757-ade6-38ee7065e57b None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:52.348892 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1f7851-c21b-4a41-aae0-45d79415c899 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:52.381507 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1980dba-bb97-43f5-94c6-e5b200bd0b5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:52.382138 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1980dba-bb97-43f5-94c6-e5b200bd0b5a None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:52.382419 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1980dba-bb97-43f5-94c6-e5b200bd0b5a None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:52.382693 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1980dba-bb97-43f5-94c6-e5b200bd0b5a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:52.392690 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1980dba-bb97-43f5-94c6-e5b200bd0b5a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:27:52.397777 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application [None req-c1980dba-bb97-43f5-94c6-e5b200bd0b5a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 17:27:52.397777 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:27:52.397777 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:27:52.397777 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:27:52.397777 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:27:52.397777 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:27:52.397777 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:27:52.397777 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:27:52.397777 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:27:52.397777 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:27:52.397777 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:27:52.397777 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:27:52.397777 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:27:52.397777 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:27:52.397777 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:27:52.397777 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:27:52.398631 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:27:52.398631 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:27:52.398631 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:27:52.398631 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:27:52.398631 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:27:52.398631 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:27:52.398631 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:27:52.398631 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:27:52.398631 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:27:52.398631 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:27:52.398631 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:27:52.398631 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application with Lock( Aug 24 17:27:52.398631 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:27:52.398631 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:27:52.398631 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:27:52.398631 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:27:52.399419 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:27:52.399419 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:27:52.399419 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:27:52.399419 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:27:52.399419 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:27:52.399419 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:27:52.399419 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:27:52.399419 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:27:52.399419 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:27:52.399419 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:27:52.399419 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 17:27:52.399419 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Aug 24 17:27:52.400007 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:52.400077 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 249/506] 10.222.0.45 () {56 vars in 1102 bytes} [Wed Aug 24 17:27:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:27:52.400077 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1980dba-bb97-43f5-94c6-e5b200bd0b5a None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:52.414520 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534c89d4-e041-4757-ade6-38ee7065e57b None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:52.419985 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:52.448246 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833eeaf8-b853-472d-9a96-9dcf6072d9e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:52.448811 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833eeaf8-b853-472d-9a96-9dcf6072d9e9 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:52.449094 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833eeaf8-b853-472d-9a96-9dcf6072d9e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:52.449368 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833eeaf8-b853-472d-9a96-9dcf6072d9e9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:52.499891 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1980dba-bb97-43f5-94c6-e5b200bd0b5a None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:52.514325 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833eeaf8-b853-472d-9a96-9dcf6072d9e9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:52.515789 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833eeaf8-b853-472d-9a96-9dcf6072d9e9 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:52.532869 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0402955c-adf8-43c5-b57a-cdba2c8d1dc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:52.533473 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0402955c-adf8-43c5-b57a-cdba2c8d1dc1 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:52.533750 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0402955c-adf8-43c5-b57a-cdba2c8d1dc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:52.534022 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0402955c-adf8-43c5-b57a-cdba2c8d1dc1 None admin] PATH_INFO: `/v3/domains` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:52.534661 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0402955c-adf8-43c5-b57a-cdba2c8d1dc1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:27:52.534962 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0402955c-adf8-43c5-b57a-cdba2c8d1dc1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:52.536313 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0402955c-adf8-43c5-b57a-cdba2c8d1dc1 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:52.547502 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 250/507] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 24 17:27:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:27:52.548277 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0402955c-adf8-43c5-b57a-cdba2c8d1dc1 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:52.602512 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:52.620331 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 258/508] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Aug 24 17:27:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 17:27:52.620331 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833eeaf8-b853-472d-9a96-9dcf6072d9e9 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:52.622868 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:52.644814 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0402955c-adf8-43c5-b57a-cdba2c8d1dc1 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:52.677862 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224c4643-30bf-4638-8108-c55805387944 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:52.678483 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224c4643-30bf-4638-8108-c55805387944 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:52.678761 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224c4643-30bf-4638-8108-c55805387944 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:52.679033 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224c4643-30bf-4638-8108-c55805387944 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:52.688727 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224c4643-30bf-4638-8108-c55805387944 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:52.690180 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224c4643-30bf-4638-8108-c55805387944 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:52.704655 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application [None req-224c4643-30bf-4638-8108-c55805387944 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:27:52.704655 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:27:52.704655 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:27:52.704655 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:27:52.704655 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:27:52.704655 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:27:52.704655 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:27:52.704655 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:27:52.704655 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:27:52.704655 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:27:52.704655 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:27:52.704655 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:27:52.704655 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:27:52.704655 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:27:52.704655 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:27:52.704655 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:27:52.705526 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:27:52.705526 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:27:52.705526 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:27:52.705526 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:27:52.705526 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:27:52.705526 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:27:52.705526 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:27:52.705526 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application with Lock( Aug 24 17:27:52.705526 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:27:52.705526 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:27:52.705526 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:27:52.705526 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:27:52.705526 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:27:52.705526 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:27:52.705526 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:27:52.705526 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:27:52.705526 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:27:52.706422 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:27:52.706422 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:27:52.706422 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Aug 24 17:27:52.706422 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 251/509] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 17:27:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:27:52.706422 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224c4643-30bf-4638-8108-c55805387944 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:52.717557 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833eeaf8-b853-472d-9a96-9dcf6072d9e9 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:52.751023 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a203c95-409d-4b4f-a00a-f7f5a14eb20c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:52.751630 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a203c95-409d-4b4f-a00a-f7f5a14eb20c None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:52.751911 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a203c95-409d-4b4f-a00a-f7f5a14eb20c None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:52.752184 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a203c95-409d-4b4f-a00a-f7f5a14eb20c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:52.807355 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224c4643-30bf-4638-8108-c55805387944 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:52.809466 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:52.821910 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a203c95-409d-4b4f-a00a-f7f5a14eb20c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:52.823878 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a203c95-409d-4b4f-a00a-f7f5a14eb20c None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:52.835284 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:52.842380 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1458616-e1cc-48c1-90ec-462d9c0b49e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:52.842987 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1458616-e1cc-48c1-90ec-462d9c0b49e8 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:52.843268 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1458616-e1cc-48c1-90ec-462d9c0b49e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:52.843540 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1458616-e1cc-48c1-90ec-462d9c0b49e8 None admin] PATH_INFO: `/v3/roles` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:52.844208 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1458616-e1cc-48c1-90ec-462d9c0b49e8 None admin] RBAC: Adding query filter params (name=service) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:27:52.844513 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1458616-e1cc-48c1-90ec-462d9c0b49e8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:52.845915 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1458616-e1cc-48c1-90ec-462d9c0b49e8 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:52.853628 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 252/510] 10.222.0.45 () {56 vars in 1103 bytes} [Wed Aug 24 17:27:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:27:52.854695 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1458616-e1cc-48c1-90ec-462d9c0b49e8 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:52.927587 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 259/511] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Aug 24 17:27:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 17:27:52.928421 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a203c95-409d-4b4f-a00a-f7f5a14eb20c None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:52.957713 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1458616-e1cc-48c1-90ec-462d9c0b49e8 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:52.990113 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee9db2c-935d-4313-92ec-2b8376217954 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:52.990732 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee9db2c-935d-4313-92ec-2b8376217954 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:52.991010 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee9db2c-935d-4313-92ec-2b8376217954 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:52.991282 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee9db2c-935d-4313-92ec-2b8376217954 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:52.998875 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee9db2c-935d-4313-92ec-2b8376217954 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:53.000340 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee9db2c-935d-4313-92ec-2b8376217954 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:53.011563 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application [None req-fee9db2c-935d-4313-92ec-2b8376217954 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 17:27:53.011563 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:27:53.011563 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:27:53.011563 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:27:53.011563 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:27:53.011563 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:27:53.011563 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:27:53.011563 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:27:53.011563 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:27:53.011563 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:27:53.011563 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:27:53.011563 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:27:53.011563 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 17:27:53.011563 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 17:27:53.011563 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 17:27:53.011563 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 17:27:53.012580 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:27:53.012580 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:27:53.012580 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 17:27:53.012580 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:27:53.012580 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 17:27:53.012580 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:27:53.012580 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:27:53.012580 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:27:53.012580 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:27:53.012580 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:27:53.012580 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:27:53.012580 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application with Lock( Aug 24 17:27:53.012580 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:27:53.012580 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:27:53.012580 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:27:53.012580 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:27:53.012580 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:27:53.012580 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:27:53.018343 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:27:53.018343 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:27:53.018343 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 17:27:53.018343 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 17:27:53.018343 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 17:27:53.018343 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 17:27:53.018343 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 17:27:53.018343 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 17:27:53.018343 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 17:27:53.018343 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Aug 24 17:27:53.018343 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 253/512] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Aug 24 17:27:52 2022] GET /identity/v3/users/cinder => generated 84 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:27:53.018343 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee9db2c-935d-4313-92ec-2b8376217954 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:53.019635 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:53.026164 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a203c95-409d-4b4f-a00a-f7f5a14eb20c None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:53.059604 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b606ab-491b-4055-843a-d995a24be283 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:53.060234 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b606ab-491b-4055-843a-d995a24be283 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:53.060513 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b606ab-491b-4055-843a-d995a24be283 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:53.060816 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b606ab-491b-4055-843a-d995a24be283 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:53.068167 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:53.070651 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b606ab-491b-4055-843a-d995a24be283 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:53.072097 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b606ab-491b-4055-843a-d995a24be283 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:53.087434 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application [None req-e8b606ab-491b-4055-843a-d995a24be283 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:27:53.087434 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:27:53.087434 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:27:53.087434 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:27:53.087434 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:27:53.087434 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:27:53.087434 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:27:53.087434 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:27:53.087434 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:27:53.087434 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:27:53.087434 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:27:53.087434 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:27:53.087434 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:27:53.087434 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:27:53.087434 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:27:53.087434 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:27:53.088334 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:27:53.088334 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:27:53.088334 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:27:53.088334 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:27:53.088334 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:27:53.088334 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:27:53.088334 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:27:53.088334 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application with Lock( Aug 24 17:27:53.088334 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:27:53.088334 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:27:53.088334 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:27:53.088334 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:27:53.088334 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:27:53.088334 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:27:53.088334 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:27:53.088334 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:27:53.088334 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:27:53.089212 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:27:53.089212 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:27:53.089212 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Aug 24 17:27:53.089913 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 260/513] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 17:27:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:27:53.090870 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b606ab-491b-4055-843a-d995a24be283 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:53.116673 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee9db2c-935d-4313-92ec-2b8376217954 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:53.150705 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3a52e7-af1b-45d1-80f4-2cda81d22578 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:53.151514 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3a52e7-af1b-45d1-80f4-2cda81d22578 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:53.151873 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3a52e7-af1b-45d1-80f4-2cda81d22578 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:53.152212 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3a52e7-af1b-45d1-80f4-2cda81d22578 None admin] PATH_INFO: `/v3/domains` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:53.153141 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3a52e7-af1b-45d1-80f4-2cda81d22578 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:27:53.153444 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3a52e7-af1b-45d1-80f4-2cda81d22578 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:53.154798 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3a52e7-af1b-45d1-80f4-2cda81d22578 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:53.166695 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 254/514] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 24 17:27:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:27:53.167509 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3a52e7-af1b-45d1-80f4-2cda81d22578 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:53.190375 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b606ab-491b-4055-843a-d995a24be283 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:53.224880 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307efec5-4038-4c6e-accd-a6f7c1eabd73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:53.225492 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307efec5-4038-4c6e-accd-a6f7c1eabd73 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:53.225773 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307efec5-4038-4c6e-accd-a6f7c1eabd73 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:53.226042 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307efec5-4038-4c6e-accd-a6f7c1eabd73 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:53.234569 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:53.236801 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307efec5-4038-4c6e-accd-a6f7c1eabd73 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:53.238254 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307efec5-4038-4c6e-accd-a6f7c1eabd73 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:53.253038 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application [None req-307efec5-4038-4c6e-accd-a6f7c1eabd73 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:27:53.253038 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:27:53.253038 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:27:53.253038 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:27:53.253038 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:27:53.253038 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:27:53.253038 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:27:53.253038 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:27:53.253038 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:27:53.253038 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:27:53.253038 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:27:53.253038 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:27:53.253038 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:27:53.253038 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:27:53.253038 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:27:53.253038 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:27:53.253953 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:27:53.253953 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:27:53.253953 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:27:53.253953 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:27:53.253953 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:27:53.253953 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:27:53.253953 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:27:53.253953 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application with Lock( Aug 24 17:27:53.253953 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:27:53.253953 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:27:53.253953 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:27:53.253953 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:27:53.253953 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:27:53.253953 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:27:53.253953 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:27:53.253953 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:27:53.253953 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:27:53.254801 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:27:53.254801 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:27:53.254801 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Aug 24 17:27:53.254801 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 261/515] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 17:27:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:27:53.255023 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307efec5-4038-4c6e-accd-a6f7c1eabd73 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:53.269724 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3a52e7-af1b-45d1-80f4-2cda81d22578 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:53.282614 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:53.304068 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8824200f-00d6-4527-8589-59869fffa222 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:53.304724 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8824200f-00d6-4527-8589-59869fffa222 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:53.305004 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8824200f-00d6-4527-8589-59869fffa222 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:53.305276 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8824200f-00d6-4527-8589-59869fffa222 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:53.315455 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8824200f-00d6-4527-8589-59869fffa222 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:53.316918 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8824200f-00d6-4527-8589-59869fffa222 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:53.331003 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application [None req-8824200f-00d6-4527-8589-59869fffa222 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:27:53.331003 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:27:53.331003 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:27:53.331003 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:27:53.331003 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:27:53.331003 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:27:53.331003 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:27:53.331003 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:27:53.331003 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:27:53.331003 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:27:53.331003 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:27:53.331003 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:27:53.331003 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:27:53.331003 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:27:53.331003 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:27:53.331003 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:27:53.331963 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:27:53.331963 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:27:53.331963 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:27:53.331963 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:27:53.331963 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:27:53.331963 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:27:53.331963 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:27:53.331963 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application with Lock( Aug 24 17:27:53.331963 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:27:53.331963 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:27:53.331963 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:27:53.331963 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:27:53.331963 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:27:53.331963 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:27:53.331963 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:27:53.331963 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:27:53.331963 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:27:53.333173 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:27:53.333173 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:27:53.333173 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Aug 24 17:27:53.333173 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 255/516] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 17:27:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:27:53.333394 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8824200f-00d6-4527-8589-59869fffa222 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:53.358783 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307efec5-4038-4c6e-accd-a6f7c1eabd73 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:53.392645 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744ecb70-e22e-4f59-83ab-5234c06b4228 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:53.393244 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744ecb70-e22e-4f59-83ab-5234c06b4228 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:53.393522 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744ecb70-e22e-4f59-83ab-5234c06b4228 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:53.393794 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744ecb70-e22e-4f59-83ab-5234c06b4228 None admin] PATH_INFO: `/v3/users` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:53.394506 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744ecb70-e22e-4f59-83ab-5234c06b4228 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:27:53.394807 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744ecb70-e22e-4f59-83ab-5234c06b4228 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:53.396226 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744ecb70-e22e-4f59-83ab-5234c06b4228 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:53.413128 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 262/517] 10.222.0.45 () {56 vars in 1137 bytes} [Wed Aug 24 17:27:53 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:27:53.413128 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744ecb70-e22e-4f59-83ab-5234c06b4228 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:53.430479 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8824200f-00d6-4527-8589-59869fffa222 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:53.462783 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:53.462924 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04e97df-f01d-4ce0-afef-f43651b8ed5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:53.464085 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04e97df-f01d-4ce0-afef-f43651b8ed5b None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:53.464305 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04e97df-f01d-4ce0-afef-f43651b8ed5b None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:53.464602 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04e97df-f01d-4ce0-afef-f43651b8ed5b None admin] PATH_INFO: `/v3/domains` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:53.465266 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04e97df-f01d-4ce0-afef-f43651b8ed5b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:27:53.465589 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04e97df-f01d-4ce0-afef-f43651b8ed5b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:53.466996 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04e97df-f01d-4ce0-afef-f43651b8ed5b None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:53.478241 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 256/518] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 24 17:27:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:27:53.478961 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04e97df-f01d-4ce0-afef-f43651b8ed5b None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:53.504226 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:53.513892 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744ecb70-e22e-4f59-83ab-5234c06b4228 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:53.547480 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c98967a-6097-4c63-b678-be2b86da56e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:53.548083 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c98967a-6097-4c63-b678-be2b86da56e0 None admin] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:53.548361 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c98967a-6097-4c63-b678-be2b86da56e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:53.548662 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c98967a-6097-4c63-b678-be2b86da56e0 None admin] PATH_INFO: `/v3/users` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:53.549287 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c98967a-6097-4c63-b678-be2b86da56e0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:53.550705 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c98967a-6097-4c63-b678-be2b86da56e0 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:53.571208 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 263/519] 10.222.0.45 () {60 vars in 1132 bytes} [Wed Aug 24 17:27:53 2022] POST /identity/v3/users => generated 247 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:27:53.572003 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c98967a-6097-4c63-b678-be2b86da56e0 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:53.594380 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04e97df-f01d-4ce0-afef-f43651b8ed5b None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:53.637684 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d779eeb-11b5-4014-a8b1-b2b67db1e7e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:53.638412 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d779eeb-11b5-4014-a8b1-b2b67db1e7e3 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:53.638696 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d779eeb-11b5-4014-a8b1-b2b67db1e7e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:53.638968 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d779eeb-11b5-4014-a8b1-b2b67db1e7e3 None admin] PATH_INFO: `/v3/domains` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:53.639630 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d779eeb-11b5-4014-a8b1-b2b67db1e7e3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:27:53.639934 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d779eeb-11b5-4014-a8b1-b2b67db1e7e3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:53.641577 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d779eeb-11b5-4014-a8b1-b2b67db1e7e3 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:53.660350 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 257/520] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 24 17:27:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:27:53.660350 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d779eeb-11b5-4014-a8b1-b2b67db1e7e3 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:53.687916 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c98967a-6097-4c63-b678-be2b86da56e0 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:53.696812 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:53.728901 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2290f683-1517-4245-8af2-f8e0b62707d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:53.729701 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2290f683-1517-4245-8af2-f8e0b62707d2 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:53.730742 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2290f683-1517-4245-8af2-f8e0b62707d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:53.731023 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2290f683-1517-4245-8af2-f8e0b62707d2 None admin] PATH_INFO: `/v3/domains` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:53.731680 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2290f683-1517-4245-8af2-f8e0b62707d2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:27:53.731985 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2290f683-1517-4245-8af2-f8e0b62707d2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:53.733398 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2290f683-1517-4245-8af2-f8e0b62707d2 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:53.747966 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 264/521] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 24 17:27:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:27:53.748816 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2290f683-1517-4245-8af2-f8e0b62707d2 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:53.762240 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:53.784816 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d779eeb-11b5-4014-a8b1-b2b67db1e7e3 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:53.822498 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9288e3f1-dcd1-4f7a-9f30-3051843f9d77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:53.823398 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9288e3f1-dcd1-4f7a-9f30-3051843f9d77 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:53.823917 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9288e3f1-dcd1-4f7a-9f30-3051843f9d77 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:53.824485 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9288e3f1-dcd1-4f7a-9f30-3051843f9d77 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:53.860051 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2290f683-1517-4245-8af2-f8e0b62707d2 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:53.894050 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b961ad8-7f2d-4198-8e5f-3c4d29552253 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:53.894959 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b961ad8-7f2d-4198-8e5f-3c4d29552253 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:53.895238 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b961ad8-7f2d-4198-8e5f-3c4d29552253 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:53.895509 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b961ad8-7f2d-4198-8e5f-3c4d29552253 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:53.897085 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9288e3f1-dcd1-4f7a-9f30-3051843f9d77 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:53.898538 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9288e3f1-dcd1-4f7a-9f30-3051843f9d77 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:53.902866 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b961ad8-7f2d-4198-8e5f-3c4d29552253 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:53.904355 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b961ad8-7f2d-4198-8e5f-3c4d29552253 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:53.918273 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application [None req-9b961ad8-7f2d-4198-8e5f-3c4d29552253 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:27:53.918273 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:27:53.918273 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:27:53.918273 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:27:53.918273 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:27:53.918273 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:27:53.918273 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:27:53.918273 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:27:53.918273 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:27:53.918273 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:27:53.918273 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:27:53.918273 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:27:53.918273 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 17:27:53.918273 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 17:27:53.918273 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 17:27:53.919158 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 17:27:53.919158 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:27:53.919158 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:27:53.919158 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:27:53.919158 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:27:53.919158 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:27:53.919158 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:27:53.919158 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:27:53.919158 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application with Lock( Aug 24 17:27:53.919158 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:27:53.919158 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:27:53.919158 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:27:53.919158 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:27:53.919158 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:27:53.919158 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:27:53.919158 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:27:53.919158 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:27:53.919988 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 17:27:53.919988 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 17:27:53.919988 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 17:27:53.919988 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 17:27:53.919988 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 17:27:53.919988 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 17:27:53.919988 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:27:53.919988 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Aug 24 17:27:53.919988 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 265/522] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Aug 24 17:27:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:27:53.920481 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b961ad8-7f2d-4198-8e5f-3c4d29552253 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:53.948059 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:53.997016 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:54.022068 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 258/523] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Aug 24 17:27:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 17:27:54.029194 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b961ad8-7f2d-4198-8e5f-3c4d29552253 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:54.029335 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9288e3f1-dcd1-4f7a-9f30-3051843f9d77 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:54.067521 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7259a32-f128-49e4-866d-46af863f92ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:54.068213 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7259a32-f128-49e4-866d-46af863f92ec None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:54.068489 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7259a32-f128-49e4-866d-46af863f92ec None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:54.068843 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7259a32-f128-49e4-866d-46af863f92ec None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:54.078338 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7259a32-f128-49e4-866d-46af863f92ec None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:54.079899 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7259a32-f128-49e4-866d-46af863f92ec None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:54.092661 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application [None req-d7259a32-f128-49e4-866d-46af863f92ec None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 17:27:54.092661 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:27:54.092661 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:27:54.092661 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:27:54.092661 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:27:54.092661 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:27:54.092661 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:27:54.092661 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:27:54.092661 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:27:54.092661 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:27:54.092661 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:27:54.092661 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:27:54.092661 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 17:27:54.092661 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 17:27:54.092661 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 17:27:54.092661 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 17:27:54.093565 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:27:54.093565 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:27:54.093565 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 17:27:54.093565 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:27:54.093565 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 17:27:54.093565 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:27:54.093565 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:27:54.093565 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:27:54.093565 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:27:54.093565 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:27:54.093565 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:27:54.093565 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application with Lock( Aug 24 17:27:54.093565 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:27:54.093565 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:27:54.093565 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:27:54.093565 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:27:54.093565 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:27:54.093565 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:27:54.094572 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:27:54.094572 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:27:54.094572 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 17:27:54.094572 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 17:27:54.094572 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 17:27:54.094572 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 17:27:54.094572 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 17:27:54.094572 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 17:27:54.094572 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 17:27:54.094572 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Aug 24 17:27:54.094572 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 266/524] 10.222.0.45 () {56 vars in 1102 bytes} [Wed Aug 24 17:27:53 2022] GET /identity/v3/users/neutron => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:27:54.094572 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7259a32-f128-49e4-866d-46af863f92ec None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:54.140669 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9288e3f1-dcd1-4f7a-9f30-3051843f9d77 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:54.176223 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fbf738-8c03-4303-8db1-555d06d42b1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:54.176892 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fbf738-8c03-4303-8db1-555d06d42b1d None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:54.177146 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fbf738-8c03-4303-8db1-555d06d42b1d None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:54.177456 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fbf738-8c03-4303-8db1-555d06d42b1d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:54.183996 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fbf738-8c03-4303-8db1-555d06d42b1d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:54.185469 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fbf738-8c03-4303-8db1-555d06d42b1d None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:54.185625 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:54.197374 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application [None req-f4fbf738-8c03-4303-8db1-555d06d42b1d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:27:54.197374 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:27:54.197374 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:27:54.197374 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:27:54.197374 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:27:54.197374 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:27:54.197374 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:27:54.197374 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:27:54.197374 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:27:54.197374 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:27:54.197374 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:27:54.197374 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:27:54.197374 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 17:27:54.197374 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 17:27:54.197374 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 17:27:54.198762 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 17:27:54.198762 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:27:54.198762 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:27:54.198762 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:27:54.198762 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:27:54.198762 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:27:54.198762 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:27:54.198762 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:27:54.198762 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application with Lock( Aug 24 17:27:54.198762 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:27:54.198762 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:27:54.198762 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:27:54.198762 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:27:54.198762 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:27:54.198762 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:27:54.198762 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:27:54.198762 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:27:54.199590 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 17:27:54.199590 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 17:27:54.199590 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 17:27:54.199590 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 17:27:54.199590 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 17:27:54.199590 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 17:27:54.199590 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:27:54.199590 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Aug 24 17:27:54.199590 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 259/525] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Aug 24 17:27:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:27:54.200036 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fbf738-8c03-4303-8db1-555d06d42b1d None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:54.207394 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7259a32-f128-49e4-866d-46af863f92ec None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:54.242084 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bdf7cd-5bd0-479d-8c8d-412b0e6d2136 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:54.242699 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bdf7cd-5bd0-479d-8c8d-412b0e6d2136 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:54.242973 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bdf7cd-5bd0-479d-8c8d-412b0e6d2136 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:54.243248 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bdf7cd-5bd0-479d-8c8d-412b0e6d2136 None admin] PATH_INFO: `/v3/projects` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:54.243941 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36bdf7cd-5bd0-479d-8c8d-412b0e6d2136 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:27:54.244243 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36bdf7cd-5bd0-479d-8c8d-412b0e6d2136 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:54.245725 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36bdf7cd-5bd0-479d-8c8d-412b0e6d2136 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:54.252638 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:54.263809 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 267/526] 10.222.0.45 () {56 vars in 1148 bytes} [Wed Aug 24 17:27:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:27:54.264564 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bdf7cd-5bd0-479d-8c8d-412b0e6d2136 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:54.316323 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fbf738-8c03-4303-8db1-555d06d42b1d None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:54.353341 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8852b220-9a78-4053-80a4-8e457bf7d295 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:54.354438 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8852b220-9a78-4053-80a4-8e457bf7d295 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:54.354722 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8852b220-9a78-4053-80a4-8e457bf7d295 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:54.354991 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8852b220-9a78-4053-80a4-8e457bf7d295 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:54.364720 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8852b220-9a78-4053-80a4-8e457bf7d295 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:54.366168 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8852b220-9a78-4053-80a4-8e457bf7d295 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:54.368691 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bdf7cd-5bd0-479d-8c8d-412b0e6d2136 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:54.379995 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application [None req-8852b220-9a78-4053-80a4-8e457bf7d295 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:27:54.379995 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:27:54.379995 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:27:54.379995 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:27:54.379995 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:27:54.379995 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:27:54.379995 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:27:54.379995 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:27:54.379995 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:27:54.379995 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:27:54.379995 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:27:54.379995 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:27:54.379995 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:27:54.379995 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:27:54.379995 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:27:54.379995 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:27:54.381099 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:27:54.381099 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:27:54.381099 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:27:54.381099 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:27:54.381099 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:27:54.381099 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:27:54.381099 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:27:54.381099 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application with Lock( Aug 24 17:27:54.381099 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:27:54.381099 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:27:54.381099 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:27:54.381099 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:27:54.381099 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:27:54.381099 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:27:54.381099 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:27:54.381099 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:27:54.381099 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:27:54.381929 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:27:54.381929 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:27:54.381929 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Aug 24 17:27:54.381929 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 260/527] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 17:27:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:27:54.382263 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8852b220-9a78-4053-80a4-8e457bf7d295 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:54.407725 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1e4b50-7b95-432b-950e-23eef7730277 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:54.408331 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1e4b50-7b95-432b-950e-23eef7730277 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:54.408644 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1e4b50-7b95-432b-950e-23eef7730277 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:54.408920 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1e4b50-7b95-432b-950e-23eef7730277 None admin] PATH_INFO: `/v3/users` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:54.409641 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1e4b50-7b95-432b-950e-23eef7730277 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:27:54.409999 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1e4b50-7b95-432b-950e-23eef7730277 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:54.411740 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1e4b50-7b95-432b-950e-23eef7730277 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:54.434279 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 268/528] 10.222.0.45 () {56 vars in 1139 bytes} [Wed Aug 24 17:27:54 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:27:54.434279 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1e4b50-7b95-432b-950e-23eef7730277 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:54.435611 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:54.494376 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:54.501589 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8852b220-9a78-4053-80a4-8e457bf7d295 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:54.533691 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1e4b50-7b95-432b-950e-23eef7730277 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:54.535757 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1528676-a36a-43e9-9764-85b7df9d42a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:54.536443 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1528676-a36a-43e9-9764-85b7df9d42a2 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:54.536918 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1528676-a36a-43e9-9764-85b7df9d42a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:54.537287 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1528676-a36a-43e9-9764-85b7df9d42a2 None admin] PATH_INFO: `/v3/projects` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:54.538082 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1528676-a36a-43e9-9764-85b7df9d42a2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:27:54.538487 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1528676-a36a-43e9-9764-85b7df9d42a2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:54.540033 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1528676-a36a-43e9-9764-85b7df9d42a2 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:54.558185 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 261/529] 10.222.0.45 () {56 vars in 1148 bytes} [Wed Aug 24 17:27:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:27:54.559174 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1528676-a36a-43e9-9764-85b7df9d42a2 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:54.574191 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f3f2c8-2c3c-429b-89a3-1fd779b0ee23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:54.574976 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f3f2c8-2c3c-429b-89a3-1fd779b0ee23 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:54.575334 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f3f2c8-2c3c-429b-89a3-1fd779b0ee23 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:54.575718 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f3f2c8-2c3c-429b-89a3-1fd779b0ee23 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:54.576517 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f3f2c8-2c3c-429b-89a3-1fd779b0ee23 None admin] RBAC: Adding query filter params (role.id=ad6a81fb5d1941888577eb0ce9474882, scope.project.id=2ad8a32bd9d2434097bd08b111914ca1, user.id=053b7ce6d6f649ed91a26e434dab805e) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:27:54.576919 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f3f2c8-2c3c-429b-89a3-1fd779b0ee23 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:54.578421 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f3f2c8-2c3c-429b-89a3-1fd779b0ee23 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:54.588595 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 269/530] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Aug 24 17:27:54 2022] GET /identity/v3/role_assignments?user.id=053b7ce6d6f649ed91a26e434dab805e&scope.project.id=2ad8a32bd9d2434097bd08b111914ca1&role.id=ad6a81fb5d1941888577eb0ce9474882 => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:27:54.588595 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f3f2c8-2c3c-429b-89a3-1fd779b0ee23 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:54.685256 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:54.690544 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1528676-a36a-43e9-9764-85b7df9d42a2 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:54.730177 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c125d161-cd1d-4ba9-9351-91a48fa25057 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:54.730893 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c125d161-cd1d-4ba9-9351-91a48fa25057 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:54.731235 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c125d161-cd1d-4ba9-9351-91a48fa25057 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:54.731584 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c125d161-cd1d-4ba9-9351-91a48fa25057 None admin] PATH_INFO: `/v3/domains` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:54.732309 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c125d161-cd1d-4ba9-9351-91a48fa25057 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:27:54.732717 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c125d161-cd1d-4ba9-9351-91a48fa25057 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:54.734188 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c125d161-cd1d-4ba9-9351-91a48fa25057 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:54.734863 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f3f2c8-2c3c-429b-89a3-1fd779b0ee23 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:54.747712 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 262/531] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 24 17:27:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:27:54.748593 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c125d161-cd1d-4ba9-9351-91a48fa25057 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:54.769893 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:54.786844 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6339199a-cbf8-4727-bd1d-1354f4f41bdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:54.787773 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6339199a-cbf8-4727-bd1d-1354f4f41bdd None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:54.789254 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6339199a-cbf8-4727-bd1d-1354f4f41bdd None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:54.789804 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6339199a-cbf8-4727-bd1d-1354f4f41bdd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:54.862268 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c125d161-cd1d-4ba9-9351-91a48fa25057 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:54.879747 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6339199a-cbf8-4727-bd1d-1354f4f41bdd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:54.881871 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6339199a-cbf8-4727-bd1d-1354f4f41bdd None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:54.899254 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa56aa1f-59d4-493e-accd-bc3b240eb323 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:54.900085 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa56aa1f-59d4-493e-accd-bc3b240eb323 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:54.900500 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa56aa1f-59d4-493e-accd-bc3b240eb323 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:54.900993 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa56aa1f-59d4-493e-accd-bc3b240eb323 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:54.901882 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa56aa1f-59d4-493e-accd-bc3b240eb323 None admin] RBAC: Adding query filter params (role.id=ad6a81fb5d1941888577eb0ce9474882, scope.project.id=2ad8a32bd9d2434097bd08b111914ca1, user.id=df5fee57539447428dd1c72eab3cfd9d) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:27:54.902334 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa56aa1f-59d4-493e-accd-bc3b240eb323 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:54.903922 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa56aa1f-59d4-493e-accd-bc3b240eb323 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:54.910260 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 263/532] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Aug 24 17:27:54 2022] GET /identity/v3/role_assignments?user.id=df5fee57539447428dd1c72eab3cfd9d&scope.project.id=2ad8a32bd9d2434097bd08b111914ca1&role.id=ad6a81fb5d1941888577eb0ce9474882 => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:27:54.911371 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa56aa1f-59d4-493e-accd-bc3b240eb323 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:54.970529 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:55.003444 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:55.020660 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 270/533] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Aug 24 17:27:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 17:27:55.029666 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6339199a-cbf8-4727-bd1d-1354f4f41bdd None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:55.042565 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa56aa1f-59d4-493e-accd-bc3b240eb323 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:55.099297 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34047803-9862-496f-9875-741a34a6275a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:55.099950 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34047803-9862-496f-9875-741a34a6275a None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:55.100228 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34047803-9862-496f-9875-741a34a6275a None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:55.100501 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34047803-9862-496f-9875-741a34a6275a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:55.119517 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34047803-9862-496f-9875-741a34a6275a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:55.121069 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34047803-9862-496f-9875-741a34a6275a None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:55.142688 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application [None req-34047803-9862-496f-9875-741a34a6275a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:27:55.142688 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:27:55.142688 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:27:55.142688 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:27:55.142688 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:27:55.142688 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:27:55.142688 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:27:55.142688 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:27:55.142688 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:27:55.142688 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:27:55.142688 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:27:55.142688 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:27:55.142688 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 17:27:55.142688 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 17:27:55.142688 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 17:27:55.143613 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 17:27:55.143613 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:27:55.143613 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:27:55.143613 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:27:55.143613 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:27:55.143613 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:27:55.143613 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:27:55.143613 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:27:55.143613 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application with Lock( Aug 24 17:27:55.143613 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:27:55.143613 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:27:55.143613 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:27:55.143613 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:27:55.143613 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:27:55.143613 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:27:55.143613 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:27:55.143613 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:27:55.144437 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 17:27:55.144437 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 17:27:55.144437 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 17:27:55.144437 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 17:27:55.144437 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 17:27:55.144437 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 17:27:55.144437 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:27:55.144437 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Aug 24 17:27:55.144437 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 264/534] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Aug 24 17:27:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:27:55.151944 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34047803-9862-496f-9875-741a34a6275a None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:55.229834 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6339199a-cbf8-4727-bd1d-1354f4f41bdd None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:55.291777 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34047803-9862-496f-9875-741a34a6275a None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:55.305952 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76178aba-6b66-4aec-b76b-395522b8f55d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:55.306504 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76178aba-6b66-4aec-b76b-395522b8f55d None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:55.306786 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76178aba-6b66-4aec-b76b-395522b8f55d None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:55.307605 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76178aba-6b66-4aec-b76b-395522b8f55d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:55.332796 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76178aba-6b66-4aec-b76b-395522b8f55d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:55.334278 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76178aba-6b66-4aec-b76b-395522b8f55d None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:55.356615 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application [None req-76178aba-6b66-4aec-b76b-395522b8f55d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:27:55.356615 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:27:55.356615 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:27:55.356615 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:27:55.356615 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:27:55.356615 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:27:55.356615 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:27:55.356615 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:27:55.356615 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:27:55.356615 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:27:55.356615 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:27:55.356615 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:27:55.356615 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:27:55.356615 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:27:55.356615 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:27:55.356615 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:27:55.357567 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:27:55.357567 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:27:55.357567 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:27:55.357567 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:27:55.357567 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:27:55.357567 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:27:55.357567 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:27:55.357567 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application with Lock( Aug 24 17:27:55.357567 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:27:55.357567 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:27:55.357567 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:27:55.357567 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:27:55.357567 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:27:55.357567 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:27:55.357567 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:27:55.357567 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:27:55.357567 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:27:55.359704 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:27:55.359704 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:27:55.359704 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Aug 24 17:27:55.359704 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 271/535] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 17:27:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 329 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:27:55.361853 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ef5803-0ffc-4b22-8cde-273444c92971 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:55.362486 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ef5803-0ffc-4b22-8cde-273444c92971 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:55.362767 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ef5803-0ffc-4b22-8cde-273444c92971 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:55.363040 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ef5803-0ffc-4b22-8cde-273444c92971 None admin] PATH_INFO: `/v3/projects` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:55.363758 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ef5803-0ffc-4b22-8cde-273444c92971 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:27:55.364064 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ef5803-0ffc-4b22-8cde-273444c92971 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:55.365550 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ef5803-0ffc-4b22-8cde-273444c92971 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:55.371282 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76178aba-6b66-4aec-b76b-395522b8f55d None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:55.381481 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 265/536] 10.222.0.45 () {56 vars in 1148 bytes} [Wed Aug 24 17:27:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:27:55.381995 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:55.396641 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ef5803-0ffc-4b22-8cde-273444c92971 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:55.420922 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:55.548658 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ef5803-0ffc-4b22-8cde-273444c92971 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:55.561013 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76178aba-6b66-4aec-b76b-395522b8f55d None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:55.603252 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a86511e-e959-46c3-a998-e084ccfcb86b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:55.605139 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a86511e-e959-46c3-a998-e084ccfcb86b None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:55.605600 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a86511e-e959-46c3-a998-e084ccfcb86b None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:55.606024 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a86511e-e959-46c3-a998-e084ccfcb86b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:55.606766 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a86511e-e959-46c3-a998-e084ccfcb86b None admin] RBAC: Adding query filter params (role.id=ad6a81fb5d1941888577eb0ce9474882, scope.project.id=2ad8a32bd9d2434097bd08b111914ca1, user.id=59be342541264942ad87f60ff4de6c8e) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:27:55.607077 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a86511e-e959-46c3-a998-e084ccfcb86b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:55.608524 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a86511e-e959-46c3-a998-e084ccfcb86b None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:55.611842 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866cb7ff-020a-46f0-883b-fbb41e95034c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:55.612471 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866cb7ff-020a-46f0-883b-fbb41e95034c None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:55.612819 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866cb7ff-020a-46f0-883b-fbb41e95034c None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:55.613101 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866cb7ff-020a-46f0-883b-fbb41e95034c None admin] PATH_INFO: `/v3/domains` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:55.613759 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866cb7ff-020a-46f0-883b-fbb41e95034c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:27:55.614067 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866cb7ff-020a-46f0-883b-fbb41e95034c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:55.615445 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866cb7ff-020a-46f0-883b-fbb41e95034c None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:55.620694 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 266/537] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Aug 24 17:27:55 2022] GET /identity/v3/role_assignments?user.id=59be342541264942ad87f60ff4de6c8e&scope.project.id=2ad8a32bd9d2434097bd08b111914ca1&role.id=ad6a81fb5d1941888577eb0ce9474882 => generated 259 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:27:55.623707 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcf8a1c-de81-497e-a5c6-05e303dfbeb4 None None] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:55.624184 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcf8a1c-de81-497e-a5c6-05e303dfbeb4 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:55.624633 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcf8a1c-de81-497e-a5c6-05e303dfbeb4 None None] PATH_INFO: `/` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:55.628697 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 267/538] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:27:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:27:55.637985 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86687603-c6ec-46e6-b97b-2c3633a80f09 None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:55.638311 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86687603-c6ec-46e6-b97b-2c3633a80f09 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:55.638534 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86687603-c6ec-46e6-b97b-2c3633a80f09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:55.640596 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 272/539] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 24 17:27:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:27:55.646299 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866cb7ff-020a-46f0-883b-fbb41e95034c None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:55.669920 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-86687603-c6ec-46e6-b97b-2c3633a80f09 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:27:55.671162 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-86687603-c6ec-46e6-b97b-2c3633a80f09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:27:55.788901 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:55.813381 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866cb7ff-020a-46f0-883b-fbb41e95034c None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:55.821492 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:55.838783 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 268/540] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:27:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:27:55.850484 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b428925b-0527-450d-82be-8a1f5ace10c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:55.851302 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b428925b-0527-450d-82be-8a1f5ace10c3 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:55.851724 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b428925b-0527-450d-82be-8a1f5ace10c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:55.852169 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b428925b-0527-450d-82be-8a1f5ace10c3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:55.866172 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b428925b-0527-450d-82be-8a1f5ace10c3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:55.867862 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b428925b-0527-450d-82be-8a1f5ace10c3 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:55.883774 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application [None req-b428925b-0527-450d-82be-8a1f5ace10c3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:27:55.883774 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:27:55.883774 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:27:55.883774 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:27:55.883774 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:27:55.883774 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:27:55.883774 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:27:55.883774 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:27:55.883774 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:27:55.883774 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:27:55.883774 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:27:55.883774 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:27:55.883774 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 17:27:55.883774 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 17:27:55.883774 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 17:27:55.884846 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 17:27:55.884846 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:27:55.884846 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:27:55.884846 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:27:55.884846 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:27:55.884846 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:27:55.884846 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:27:55.884846 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:27:55.884846 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application with Lock( Aug 24 17:27:55.884846 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:27:55.884846 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:27:55.884846 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:27:55.884846 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:27:55.884846 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:27:55.884846 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:27:55.884846 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:27:55.884846 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:27:55.885895 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 17:27:55.885895 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 17:27:55.885895 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 17:27:55.885895 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 17:27:55.885895 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 17:27:55.885895 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 17:27:55.885895 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:27:55.885895 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Aug 24 17:27:55.887867 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 273/541] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Aug 24 17:27:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:27:55.893598 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86687603-c6ec-46e6-b97b-2c3633a80f09 None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:56.018364 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86687603-c6ec-46e6-b97b-2c3633a80f09 None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:56.063628 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42992ede-247f-4fd7-b1f9-77cc4477634a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:56.064461 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42992ede-247f-4fd7-b1f9-77cc4477634a None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:56.064911 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42992ede-247f-4fd7-b1f9-77cc4477634a None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:56.065333 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42992ede-247f-4fd7-b1f9-77cc4477634a None admin] PATH_INFO: `/v3/projects` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:56.066196 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42992ede-247f-4fd7-b1f9-77cc4477634a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:27:56.066643 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42992ede-247f-4fd7-b1f9-77cc4477634a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:56.068278 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42992ede-247f-4fd7-b1f9-77cc4477634a None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:56.080972 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 269/542] 10.222.0.45 () {56 vars in 1148 bytes} [Wed Aug 24 17:27:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:27:56.086973 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b428925b-0527-450d-82be-8a1f5ace10c3 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:56.151201 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42992ede-247f-4fd7-b1f9-77cc4477634a None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:56.238516 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b428925b-0527-450d-82be-8a1f5ace10c3 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:56.240176 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:56.292585 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da0ea7e-07c7-402a-b9e1-436a4dca66b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:56.292585 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da0ea7e-07c7-402a-b9e1-436a4dca66b9 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:56.292585 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da0ea7e-07c7-402a-b9e1-436a4dca66b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:56.292585 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da0ea7e-07c7-402a-b9e1-436a4dca66b9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:56.292996 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da0ea7e-07c7-402a-b9e1-436a4dca66b9 None admin] RBAC: Adding query filter params (role.id=ad6a81fb5d1941888577eb0ce9474882, scope.project.id=2ad8a32bd9d2434097bd08b111914ca1, user.id=43ad98a250a144099ccd18378aedd4b5) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:27:56.292996 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da0ea7e-07c7-402a-b9e1-436a4dca66b9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:56.296060 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da0ea7e-07c7-402a-b9e1-436a4dca66b9 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:56.303238 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42992ede-247f-4fd7-b1f9-77cc4477634a None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:56.308804 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 274/543] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Aug 24 17:27:56 2022] GET /identity/v3/role_assignments?user.id=43ad98a250a144099ccd18378aedd4b5&scope.project.id=2ad8a32bd9d2434097bd08b111914ca1&role.id=ad6a81fb5d1941888577eb0ce9474882 => generated 259 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:27:56.358968 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c262e037-6e29-4b91-a6eb-cc7549ddd62d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:56.359533 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c262e037-6e29-4b91-a6eb-cc7549ddd62d None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:56.359816 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c262e037-6e29-4b91-a6eb-cc7549ddd62d None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:56.360090 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c262e037-6e29-4b91-a6eb-cc7549ddd62d None admin] PATH_INFO: `/v3/roles/service` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:56.379138 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c262e037-6e29-4b91-a6eb-cc7549ddd62d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:27:56.389013 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application [None req-c262e037-6e29-4b91-a6eb-cc7549ddd62d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 17:27:56.389013 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:27:56.389013 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:27:56.389013 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:27:56.389013 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:27:56.389013 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:27:56.389013 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:27:56.389013 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:27:56.389013 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:27:56.389013 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:27:56.389013 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:27:56.389013 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:27:56.389013 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:27:56.389013 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:27:56.389013 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:27:56.389013 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:27:56.389945 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:27:56.389945 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:27:56.389945 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:27:56.389945 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:27:56.389945 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:27:56.389945 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:27:56.389945 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:27:56.389945 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:27:56.389945 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:27:56.389945 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:27:56.389945 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:27:56.389945 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application with Lock( Aug 24 17:27:56.389945 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:27:56.389945 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:27:56.389945 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:27:56.389945 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:27:56.390711 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:27:56.390711 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:27:56.390711 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:27:56.390711 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:27:56.390711 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:27:56.390711 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:27:56.390711 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:27:56.390711 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:27:56.390711 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:27:56.390711 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:27:56.390711 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 17:27:56.390711 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Aug 24 17:27:56.390711 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 270/544] 10.222.0.45 () {56 vars in 1102 bytes} [Wed Aug 24 17:27:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 17:27:56.396234 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da0ea7e-07c7-402a-b9e1-436a4dca66b9 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:56.405859 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:56.550982 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da0ea7e-07c7-402a-b9e1-436a4dca66b9 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:56.594910 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b18692e-eeb7-4470-b286-cf00ae4652c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:56.595526 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b18692e-eeb7-4470-b286-cf00ae4652c9 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:56.595805 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b18692e-eeb7-4470-b286-cf00ae4652c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:56.596079 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b18692e-eeb7-4470-b286-cf00ae4652c9 None admin] PATH_INFO: `/v3/roles` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:56.596774 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b18692e-eeb7-4470-b286-cf00ae4652c9 None admin] RBAC: Adding query filter params (name=service) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:27:56.597077 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b18692e-eeb7-4470-b286-cf00ae4652c9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:56.598445 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b18692e-eeb7-4470-b286-cf00ae4652c9 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:56.612179 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 275/545] 10.222.0.45 () {56 vars in 1103 bytes} [Wed Aug 24 17:27:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:27:56.617309 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c262e037-6e29-4b91-a6eb-cc7549ddd62d None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:56.759066 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c262e037-6e29-4b91-a6eb-cc7549ddd62d None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:56.780131 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:56.813252 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23912bb1-7f57-40b4-91c5-a74feb069c05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:56.813898 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23912bb1-7f57-40b4-91c5-a74feb069c05 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:56.814183 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23912bb1-7f57-40b4-91c5-a74feb069c05 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:56.814458 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23912bb1-7f57-40b4-91c5-a74feb069c05 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:56.835484 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23912bb1-7f57-40b4-91c5-a74feb069c05 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:56.837046 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23912bb1-7f57-40b4-91c5-a74feb069c05 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:56.857545 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application [None req-23912bb1-7f57-40b4-91c5-a74feb069c05 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:27:56.857545 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:27:56.857545 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:27:56.857545 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:27:56.857545 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:27:56.857545 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:27:56.857545 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:27:56.857545 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:27:56.857545 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:27:56.857545 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:27:56.857545 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:27:56.857545 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:27:56.857545 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:27:56.857545 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:27:56.857545 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:27:56.857545 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:27:56.858486 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:27:56.858486 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:27:56.858486 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:27:56.858486 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:27:56.858486 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:27:56.858486 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:27:56.858486 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:27:56.858486 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application with Lock( Aug 24 17:27:56.858486 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:27:56.858486 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:27:56.858486 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:27:56.858486 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:27:56.858486 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:27:56.858486 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:27:56.858486 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:27:56.858486 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:27:56.858486 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:27:56.859321 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:27:56.859321 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:27:56.859321 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Aug 24 17:27:56.859321 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 271/546] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 17:27:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:27:56.864674 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b18692e-eeb7-4470-b286-cf00ae4652c9 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:56.960428 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:57.009387 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b18692e-eeb7-4470-b286-cf00ae4652c9 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:57.063002 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07200ff5-d75f-41e1-b17a-7892c792cc82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:57.063002 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07200ff5-d75f-41e1-b17a-7892c792cc82 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:57.063002 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07200ff5-d75f-41e1-b17a-7892c792cc82 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:57.063002 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07200ff5-d75f-41e1-b17a-7892c792cc82 None admin] PATH_INFO: `/v3/domains` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:57.063409 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07200ff5-d75f-41e1-b17a-7892c792cc82 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:27:57.063409 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07200ff5-d75f-41e1-b17a-7892c792cc82 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:57.063409 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07200ff5-d75f-41e1-b17a-7892c792cc82 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:57.079042 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 276/547] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 24 17:27:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:27:57.084257 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23912bb1-7f57-40b4-91c5-a74feb069c05 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:57.130156 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3e940c-1325-48d7-8aea-8823a4d587ab None None] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:57.130341 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3e940c-1325-48d7-8aea-8823a4d587ab None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:57.130626 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3e940c-1325-48d7-8aea-8823a4d587ab None None] PATH_INFO: `/` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:57.132650 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 277/548] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:27:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 17:27:57.137658 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7698b188-db3a-4c0f-a258-727e4aadf5b2 None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:57.137937 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7698b188-db3a-4c0f-a258-727e4aadf5b2 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:57.138269 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7698b188-db3a-4c0f-a258-727e4aadf5b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:57.158069 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-7698b188-db3a-4c0f-a258-727e4aadf5b2 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:27:57.159268 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-7698b188-db3a-4c0f-a258-727e4aadf5b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:27:57.238313 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23912bb1-7f57-40b4-91c5-a74feb069c05 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:57.273703 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78620c12-32c7-4061-9d8d-463be13bd20c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:57.274722 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78620c12-32c7-4061-9d8d-463be13bd20c None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:57.275202 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78620c12-32c7-4061-9d8d-463be13bd20c None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:57.275772 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78620c12-32c7-4061-9d8d-463be13bd20c None admin] PATH_INFO: `/v3/users/swift` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:57.284747 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78620c12-32c7-4061-9d8d-463be13bd20c None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:57.286660 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78620c12-32c7-4061-9d8d-463be13bd20c None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:57.288596 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 278/549] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:27:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 17:27:57.298430 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application [None req-78620c12-32c7-4061-9d8d-463be13bd20c None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 17:27:57.298430 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:27:57.298430 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:27:57.298430 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:27:57.298430 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:27:57.298430 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:27:57.298430 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:27:57.298430 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:27:57.298430 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:27:57.298430 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:27:57.298430 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:27:57.298430 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:27:57.298430 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 17:27:57.298430 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 17:27:57.298430 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 17:27:57.298430 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 17:27:57.299327 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:27:57.299327 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:27:57.299327 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 17:27:57.299327 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:27:57.299327 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 17:27:57.299327 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:27:57.299327 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:27:57.299327 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:27:57.299327 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:27:57.299327 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:27:57.299327 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:27:57.299327 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application with Lock( Aug 24 17:27:57.299327 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:27:57.299327 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:27:57.299327 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:27:57.299327 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:27:57.299327 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:27:57.299327 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:27:57.300952 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:27:57.300952 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:27:57.300952 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 17:27:57.300952 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 17:27:57.300952 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 17:27:57.300952 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 17:27:57.300952 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 17:27:57.300952 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 17:27:57.300952 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 17:27:57.300952 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Aug 24 17:27:57.300952 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 272/550] 10.222.0.45 () {56 vars in 1096 bytes} [Wed Aug 24 17:27:57 2022] GET /identity/v3/users/swift => generated 83 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:27:57.307690 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7698b188-db3a-4c0f-a258-727e4aadf5b2 None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:57.331970 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca79b2e3-737d-46a6-b01e-3cdb982db2af None None] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:57.332471 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca79b2e3-737d-46a6-b01e-3cdb982db2af None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:57.332544 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca79b2e3-737d-46a6-b01e-3cdb982db2af None None] PATH_INFO: `/` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:57.335028 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 273/551] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:27:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 17:27:57.344574 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161a9d2c-6131-415b-8d5b-d11e00d011df None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:57.344574 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161a9d2c-6131-415b-8d5b-d11e00d011df None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:57.344574 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161a9d2c-6131-415b-8d5b-d11e00d011df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:57.358892 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-161a9d2c-6131-415b-8d5b-d11e00d011df None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:27:57.360184 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-161a9d2c-6131-415b-8d5b-d11e00d011df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:27:57.381405 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:57.428651 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:57.454799 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 274/552] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:27:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:27:57.493342 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7698b188-db3a-4c0f-a258-727e4aadf5b2 None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:57.547621 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ecf232-2cc4-4b2a-9991-075792083462 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:57.548238 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ecf232-2cc4-4b2a-9991-075792083462 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:57.548517 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ecf232-2cc4-4b2a-9991-075792083462 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:57.548824 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ecf232-2cc4-4b2a-9991-075792083462 None admin] PATH_INFO: `/v3/users` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:57.549619 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ecf232-2cc4-4b2a-9991-075792083462 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:27:57.549933 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ecf232-2cc4-4b2a-9991-075792083462 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:57.551370 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ecf232-2cc4-4b2a-9991-075792083462 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:57.570128 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 279/553] 10.222.0.45 () {56 vars in 1135 bytes} [Wed Aug 24 17:27:57 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:27:57.576136 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161a9d2c-6131-415b-8d5b-d11e00d011df None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:57.636412 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ecf232-2cc4-4b2a-9991-075792083462 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:57.719441 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161a9d2c-6131-415b-8d5b-d11e00d011df None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:57.756219 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ba20b8-8652-4e8e-96be-e8c0c4686495 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:57.756861 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ba20b8-8652-4e8e-96be-e8c0c4686495 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:57.757140 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ba20b8-8652-4e8e-96be-e8c0c4686495 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:57.757412 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ba20b8-8652-4e8e-96be-e8c0c4686495 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:57.791988 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:57.810658 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ecf232-2cc4-4b2a-9991-075792083462 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:57.829848 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:57.852539 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3f51dc-fb07-4fb0-8d84-69086ade2ec7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:57.853197 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3f51dc-fb07-4fb0-8d84-69086ade2ec7 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:57.853470 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3f51dc-fb07-4fb0-8d84-69086ade2ec7 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:57.853744 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3f51dc-fb07-4fb0-8d84-69086ade2ec7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:57.857764 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ba20b8-8652-4e8e-96be-e8c0c4686495 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:57.859402 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ba20b8-8652-4e8e-96be-e8c0c4686495 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:57.865054 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3f51dc-fb07-4fb0-8d84-69086ade2ec7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:27:57.870453 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application [None req-0c3f51dc-fb07-4fb0-8d84-69086ade2ec7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 17:27:57.870453 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:27:57.870453 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:27:57.870453 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:27:57.870453 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:27:57.870453 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:27:57.870453 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:27:57.870453 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:27:57.870453 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:27:57.870453 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:27:57.870453 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:27:57.870453 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:27:57.870453 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:27:57.870453 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:27:57.870453 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:27:57.870453 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:27:57.871462 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:27:57.871462 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:27:57.871462 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:27:57.871462 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:27:57.871462 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:27:57.871462 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:27:57.871462 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:27:57.871462 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:27:57.871462 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:27:57.871462 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:27:57.871462 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:27:57.871462 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application with Lock( Aug 24 17:27:57.871462 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:27:57.871462 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:27:57.871462 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:27:57.871462 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:27:57.872272 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:27:57.872272 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:27:57.872272 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:27:57.872272 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:27:57.872272 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:27:57.872272 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:27:57.872272 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:27:57.872272 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:27:57.872272 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:27:57.872272 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:27:57.872272 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 17:27:57.872272 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Aug 24 17:27:57.872272 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 280/554] 10.222.0.45 () {56 vars in 1102 bytes} [Wed Aug 24 17:27:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 17:27:57.872989 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3f51dc-fb07-4fb0-8d84-69086ade2ec7 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:57.974360 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3f51dc-fb07-4fb0-8d84-69086ade2ec7 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:57.980263 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 275/555] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Aug 24 17:27:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 17:27:57.980263 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ba20b8-8652-4e8e-96be-e8c0c4686495 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:58.015706 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19b5581-ec2f-44ca-b146-aa4696c36954 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:58.016326 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19b5581-ec2f-44ca-b146-aa4696c36954 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:58.016891 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19b5581-ec2f-44ca-b146-aa4696c36954 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:58.016891 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19b5581-ec2f-44ca-b146-aa4696c36954 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:58.027255 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19b5581-ec2f-44ca-b146-aa4696c36954 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:27:58.032267 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application [None req-c19b5581-ec2f-44ca-b146-aa4696c36954 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 17:27:58.032267 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:27:58.032267 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:27:58.032267 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:27:58.032267 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:27:58.032267 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:27:58.032267 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:27:58.032267 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:27:58.032267 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:27:58.032267 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:27:58.032267 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:27:58.032267 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:27:58.032267 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:27:58.032267 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:27:58.032267 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:27:58.032267 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:27:58.033456 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:27:58.033456 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:27:58.033456 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:27:58.033456 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:27:58.033456 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:27:58.033456 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:27:58.033456 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:27:58.033456 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:27:58.033456 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:27:58.033456 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:27:58.033456 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:27:58.033456 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application with Lock( Aug 24 17:27:58.033456 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:27:58.033456 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:27:58.033456 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:27:58.033456 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:27:58.034322 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:27:58.034322 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:27:58.034322 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:27:58.034322 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:27:58.034322 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:27:58.034322 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:27:58.034322 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:27:58.034322 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:27:58.034322 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:27:58.034322 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:27:58.034322 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 17:27:58.034322 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Aug 24 17:27:58.034322 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 281/556] 10.222.0.45 () {56 vars in 1102 bytes} [Wed Aug 24 17:27:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:27:58.034990 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9e4663-d4e3-474b-b923-5906feef6466 None None] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:58.034990 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9e4663-d4e3-474b-b923-5906feef6466 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:58.035117 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9e4663-d4e3-474b-b923-5906feef6466 None None] PATH_INFO: `/` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:58.036438 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 282/557] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:27:58 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:27:58.037083 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9e4663-d4e3-474b-b923-5906feef6466 None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:58.043439 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:58.076692 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:58.091850 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ba20b8-8652-4e8e-96be-e8c0c4686495 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:58.128684 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fff030-0fe5-4ee7-8293-17f678bfb742 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:58.129290 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fff030-0fe5-4ee7-8293-17f678bfb742 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:58.129565 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fff030-0fe5-4ee7-8293-17f678bfb742 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:58.129836 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fff030-0fe5-4ee7-8293-17f678bfb742 None admin] PATH_INFO: `/v3/roles` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:58.130499 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fff030-0fe5-4ee7-8293-17f678bfb742 None admin] RBAC: Adding query filter params (name=service) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:27:58.130799 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fff030-0fe5-4ee7-8293-17f678bfb742 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:58.132183 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fff030-0fe5-4ee7-8293-17f678bfb742 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:58.140419 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 276/558] 10.222.0.45 () {56 vars in 1103 bytes} [Wed Aug 24 17:27:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:27:58.145636 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fff030-0fe5-4ee7-8293-17f678bfb742 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:58.147362 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9e4663-d4e3-474b-b923-5906feef6466 None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:58.186368 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822d8c52-8d94-457b-abce-09ab6303879a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:58.187007 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822d8c52-8d94-457b-abce-09ab6303879a None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:58.187291 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822d8c52-8d94-457b-abce-09ab6303879a None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:58.187569 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822d8c52-8d94-457b-abce-09ab6303879a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:58.200387 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822d8c52-8d94-457b-abce-09ab6303879a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:58.202338 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822d8c52-8d94-457b-abce-09ab6303879a None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:58.217414 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application [None req-822d8c52-8d94-457b-abce-09ab6303879a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:27:58.217414 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:27:58.217414 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:27:58.217414 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:27:58.217414 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:27:58.217414 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:27:58.217414 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:27:58.217414 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:27:58.217414 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:27:58.217414 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:27:58.217414 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:27:58.217414 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:27:58.217414 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:27:58.217414 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:27:58.217414 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:27:58.217414 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:27:58.218893 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:27:58.218893 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:27:58.218893 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:27:58.218893 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:27:58.218893 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:27:58.218893 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:27:58.218893 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:27:58.218893 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application with Lock( Aug 24 17:27:58.218893 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:27:58.218893 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:27:58.218893 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:27:58.218893 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:27:58.218893 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:27:58.218893 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:27:58.218893 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:27:58.218893 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:27:58.218893 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:27:58.219780 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:27:58.219780 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:27:58.219780 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Aug 24 17:27:58.219780 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 283/559] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 17:27:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:27:58.219780 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fa3840-eb95-43d5-98ae-402c33d27a83 None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:58.220051 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fa3840-eb95-43d5-98ae-402c33d27a83 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:58.220723 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fa3840-eb95-43d5-98ae-402c33d27a83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:58.240449 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-a1fa3840-eb95-43d5-98ae-402c33d27a83 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:27:58.241932 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-a1fa3840-eb95-43d5-98ae-402c33d27a83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:27:58.259921 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fff030-0fe5-4ee7-8293-17f678bfb742 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:58.278875 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:58.298676 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c37c92b-8e1b-4b0d-829a-62bab7469e0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:58.299293 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c37c92b-8e1b-4b0d-829a-62bab7469e0f None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:58.299569 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c37c92b-8e1b-4b0d-829a-62bab7469e0f None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:58.299846 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c37c92b-8e1b-4b0d-829a-62bab7469e0f None admin] PATH_INFO: `/v3/roles` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:58.300516 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c37c92b-8e1b-4b0d-829a-62bab7469e0f None admin] RBAC: Adding query filter params (name=service) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:27:58.300851 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c37c92b-8e1b-4b0d-829a-62bab7469e0f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:58.302238 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c37c92b-8e1b-4b0d-829a-62bab7469e0f None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:58.315249 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 277/560] 10.222.0.45 () {56 vars in 1103 bytes} [Wed Aug 24 17:27:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:27:58.324649 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 284/561] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:27:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:27:58.325093 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c37c92b-8e1b-4b0d-829a-62bab7469e0f None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:58.325194 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fa3840-eb95-43d5-98ae-402c33d27a83 None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:58.342971 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:58.434186 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c37c92b-8e1b-4b0d-829a-62bab7469e0f None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:58.439625 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fa3840-eb95-43d5-98ae-402c33d27a83 None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:58.469802 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a65cd9-2f28-47ce-a93f-7e9675258281 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:58.470435 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a65cd9-2f28-47ce-a93f-7e9675258281 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:58.470716 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a65cd9-2f28-47ce-a93f-7e9675258281 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:58.470989 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a65cd9-2f28-47ce-a93f-7e9675258281 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:58.475257 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d43f5e-1056-4ef1-866c-5f33bf16a589 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:58.475876 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d43f5e-1056-4ef1-866c-5f33bf16a589 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:58.476156 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d43f5e-1056-4ef1-866c-5f33bf16a589 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:58.476431 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d43f5e-1056-4ef1-866c-5f33bf16a589 None admin] PATH_INFO: `/v3/domains` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:58.477172 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d43f5e-1056-4ef1-866c-5f33bf16a589 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:27:58.477535 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d43f5e-1056-4ef1-866c-5f33bf16a589 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:58.479784 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d43f5e-1056-4ef1-866c-5f33bf16a589 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:58.482897 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a65cd9-2f28-47ce-a93f-7e9675258281 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:58.484353 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a65cd9-2f28-47ce-a93f-7e9675258281 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:58.496272 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 285/562] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 24 17:27:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:27:58.497192 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d43f5e-1056-4ef1-866c-5f33bf16a589 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:58.499253 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application [None req-17a65cd9-2f28-47ce-a93f-7e9675258281 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:27:58.499253 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:27:58.499253 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:27:58.499253 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:27:58.499253 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:27:58.499253 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:27:58.499253 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:27:58.499253 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:27:58.499253 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:27:58.499253 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:27:58.499253 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:27:58.499253 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:27:58.499253 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:27:58.499253 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:27:58.499253 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:27:58.499253 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:27:58.500167 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:27:58.500167 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:27:58.500167 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:27:58.500167 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:27:58.500167 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:27:58.500167 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:27:58.500167 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:27:58.500167 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application with Lock( Aug 24 17:27:58.500167 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:27:58.500167 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:27:58.500167 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:27:58.500167 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:27:58.500167 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:27:58.500167 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:27:58.500167 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:27:58.500167 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:27:58.500167 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:27:58.501108 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:27:58.501108 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:27:58.501108 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Aug 24 17:27:58.501108 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 278/563] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 17:27:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:27:58.501349 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a65cd9-2f28-47ce-a93f-7e9675258281 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:58.534695 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:58.584142 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:58.610010 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a65cd9-2f28-47ce-a93f-7e9675258281 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:58.610631 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d43f5e-1056-4ef1-866c-5f33bf16a589 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:58.644040 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343ab01c-d280-4e58-b169-0a87647b7e38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:58.644698 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343ab01c-d280-4e58-b169-0a87647b7e38 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:58.644980 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343ab01c-d280-4e58-b169-0a87647b7e38 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:58.645252 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343ab01c-d280-4e58-b169-0a87647b7e38 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:58.649150 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6498de45-033c-4575-8b6f-1fcee8d48696 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:58.649946 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6498de45-033c-4575-8b6f-1fcee8d48696 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:58.650376 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6498de45-033c-4575-8b6f-1fcee8d48696 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:58.650845 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6498de45-033c-4575-8b6f-1fcee8d48696 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:58.651871 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343ab01c-d280-4e58-b169-0a87647b7e38 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:58.653358 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343ab01c-d280-4e58-b169-0a87647b7e38 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:58.662264 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6498de45-033c-4575-8b6f-1fcee8d48696 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:58.663887 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6498de45-033c-4575-8b6f-1fcee8d48696 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:58.664266 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application [None req-343ab01c-d280-4e58-b169-0a87647b7e38 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:27:58.664266 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:27:58.664266 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:27:58.664266 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:27:58.664266 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:27:58.664266 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:27:58.664266 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:27:58.664266 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:27:58.664266 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:27:58.664266 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:27:58.664266 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:27:58.664266 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:27:58.664266 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 17:27:58.664266 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 17:27:58.664266 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 17:27:58.665405 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 17:27:58.665405 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:27:58.665405 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:27:58.665405 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:27:58.665405 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:27:58.665405 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:27:58.665405 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:27:58.665405 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:27:58.665405 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application with Lock( Aug 24 17:27:58.665405 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:27:58.665405 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:27:58.665405 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:27:58.665405 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:27:58.665405 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:27:58.665405 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:27:58.665405 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:27:58.665405 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:27:58.666992 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 17:27:58.666992 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 17:27:58.666992 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 17:27:58.666992 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 17:27:58.666992 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 17:27:58.666992 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 17:27:58.666992 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:27:58.666992 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Aug 24 17:27:58.666992 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 279/564] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Aug 24 17:27:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:27:58.666992 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343ab01c-d280-4e58-b169-0a87647b7e38 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:58.686773 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application [None req-6498de45-033c-4575-8b6f-1fcee8d48696 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:27:58.686773 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:27:58.686773 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:27:58.686773 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:27:58.686773 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:27:58.686773 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:27:58.686773 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:27:58.686773 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:27:58.686773 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:27:58.686773 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:27:58.686773 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:27:58.686773 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:27:58.686773 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:27:58.686773 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:27:58.686773 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:27:58.686773 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:27:58.687839 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:27:58.687839 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:27:58.687839 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:27:58.687839 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:27:58.687839 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:27:58.687839 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:27:58.687839 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:27:58.687839 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application with Lock( Aug 24 17:27:58.687839 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:27:58.687839 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:27:58.687839 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:27:58.687839 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:27:58.687839 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:27:58.687839 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:27:58.687839 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:27:58.687839 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:27:58.687839 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:27:58.688915 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:27:58.688915 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:27:58.688915 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Aug 24 17:27:58.688915 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 286/565] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 17:27:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:27:58.688915 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6498de45-033c-4575-8b6f-1fcee8d48696 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:58.768202 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:58.778094 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343ab01c-d280-4e58-b169-0a87647b7e38 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:58.792004 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6498de45-033c-4575-8b6f-1fcee8d48696 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:58.812477 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8178d6c9-f30d-42c8-8059-4bf760b8d32f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:58.813108 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8178d6c9-f30d-42c8-8059-4bf760b8d32f None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:58.813387 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8178d6c9-f30d-42c8-8059-4bf760b8d32f None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:58.813661 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8178d6c9-f30d-42c8-8059-4bf760b8d32f None admin] PATH_INFO: `/v3/domains` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:58.814303 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8178d6c9-f30d-42c8-8059-4bf760b8d32f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:27:58.814604 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8178d6c9-f30d-42c8-8059-4bf760b8d32f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:58.815963 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8178d6c9-f30d-42c8-8059-4bf760b8d32f None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:58.818213 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:58.827189 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4263b463-357d-4eb9-aa4b-10f8c5a4cf7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:58.827816 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4263b463-357d-4eb9-aa4b-10f8c5a4cf7f None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:58.828098 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4263b463-357d-4eb9-aa4b-10f8c5a4cf7f None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:58.828375 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4263b463-357d-4eb9-aa4b-10f8c5a4cf7f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:58.828512 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 280/566] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 24 17:27:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:27:58.829362 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8178d6c9-f30d-42c8-8059-4bf760b8d32f None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:58.841334 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4263b463-357d-4eb9-aa4b-10f8c5a4cf7f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:27:58.846383 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application [None req-4263b463-357d-4eb9-aa4b-10f8c5a4cf7f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 17:27:58.846383 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:27:58.846383 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:27:58.846383 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:27:58.846383 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:27:58.846383 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:27:58.846383 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:27:58.846383 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:27:58.846383 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:27:58.846383 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:27:58.846383 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:27:58.846383 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:27:58.846383 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:27:58.846383 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:27:58.846383 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:27:58.846383 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:27:58.847397 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:27:58.847397 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:27:58.847397 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:27:58.847397 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:27:58.847397 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:27:58.847397 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:27:58.847397 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:27:58.847397 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:27:58.847397 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:27:58.847397 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:27:58.847397 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:27:58.847397 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application with Lock( Aug 24 17:27:58.847397 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:27:58.847397 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:27:58.847397 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:27:58.847397 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:27:58.848209 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:27:58.848209 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:27:58.848209 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:27:58.848209 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:27:58.848209 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:27:58.848209 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:27:58.848209 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:27:58.848209 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:27:58.848209 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:27:58.848209 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:27:58.848209 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 17:27:58.848209 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Aug 24 17:27:58.848209 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 287/567] 10.222.0.45 () {56 vars in 1102 bytes} [Wed Aug 24 17:27:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:27:58.848940 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c6dd3c-1dde-483d-9d7b-9f4e02220634 None None] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:58.848940 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c6dd3c-1dde-483d-9d7b-9f4e02220634 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:58.849486 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c6dd3c-1dde-483d-9d7b-9f4e02220634 None None] PATH_INFO: `/` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:58.850327 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 288/568] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:27:58 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:27:58.851008 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c6dd3c-1dde-483d-9d7b-9f4e02220634 None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:58.935121 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8178d6c9-f30d-42c8-8059-4bf760b8d32f None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:58.950340 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c6dd3c-1dde-483d-9d7b-9f4e02220634 None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:58.967808 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63ea7a6-4f09-4363-a48e-082360d9e9c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:58.968407 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63ea7a6-4f09-4363-a48e-082360d9e9c7 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:58.968721 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63ea7a6-4f09-4363-a48e-082360d9e9c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:58.968994 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63ea7a6-4f09-4363-a48e-082360d9e9c7 None admin] PATH_INFO: `/v3/projects` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:58.969686 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63ea7a6-4f09-4363-a48e-082360d9e9c7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:27:58.969991 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63ea7a6-4f09-4363-a48e-082360d9e9c7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:58.971415 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63ea7a6-4f09-4363-a48e-082360d9e9c7 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:58.983355 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 281/569] 10.222.0.45 () {56 vars in 1148 bytes} [Wed Aug 24 17:27:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:27:58.983488 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fec886b-5a86-4997-8818-43a77b1d6ab5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:58.984004 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fec886b-5a86-4997-8818-43a77b1d6ab5 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:58.984193 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63ea7a6-4f09-4363-a48e-082360d9e9c7 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:58.984283 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fec886b-5a86-4997-8818-43a77b1d6ab5 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:58.984569 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fec886b-5a86-4997-8818-43a77b1d6ab5 None admin] PATH_INFO: `/v3/domains` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:58.985210 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fec886b-5a86-4997-8818-43a77b1d6ab5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:27:58.985512 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fec886b-5a86-4997-8818-43a77b1d6ab5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:58.986897 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fec886b-5a86-4997-8818-43a77b1d6ab5 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:58.998884 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 289/570] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 24 17:27:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:27:58.999750 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fec886b-5a86-4997-8818-43a77b1d6ab5 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:59.001094 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:59.049574 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:59.084035 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63ea7a6-4f09-4363-a48e-082360d9e9c7 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:59.098004 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fec886b-5a86-4997-8818-43a77b1d6ab5 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:59.117622 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9d6d7b-e540-4f43-8c5e-fc0ce0a1135c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:59.118337 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9d6d7b-e540-4f43-8c5e-fc0ce0a1135c None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:59.118647 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9d6d7b-e540-4f43-8c5e-fc0ce0a1135c None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:59.118919 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9d6d7b-e540-4f43-8c5e-fc0ce0a1135c None admin] PATH_INFO: `/v3/users/nova` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:59.126052 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9d6d7b-e540-4f43-8c5e-fc0ce0a1135c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:59.127503 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9d6d7b-e540-4f43-8c5e-fc0ce0a1135c None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:59.130704 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9f3a1c-10e5-4f68-a7ac-e930ed7bc0ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:59.131299 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9f3a1c-10e5-4f68-a7ac-e930ed7bc0ad None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:59.131579 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9f3a1c-10e5-4f68-a7ac-e930ed7bc0ad None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:59.131855 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9f3a1c-10e5-4f68-a7ac-e930ed7bc0ad None admin] PATH_INFO: `/v3/roles` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:59.132510 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9f3a1c-10e5-4f68-a7ac-e930ed7bc0ad None admin] RBAC: Adding query filter params (name=service) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:27:59.132842 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9f3a1c-10e5-4f68-a7ac-e930ed7bc0ad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:59.134200 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9f3a1c-10e5-4f68-a7ac-e930ed7bc0ad None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:59.138176 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application [None req-ac9d6d7b-e540-4f43-8c5e-fc0ce0a1135c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 17:27:59.138176 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:27:59.138176 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:27:59.138176 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:27:59.138176 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:27:59.138176 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:27:59.138176 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:27:59.138176 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:27:59.138176 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:27:59.138176 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:27:59.138176 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:27:59.138176 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:27:59.138176 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 17:27:59.138176 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 17:27:59.138176 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 17:27:59.138176 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 17:27:59.139199 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:27:59.139199 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:27:59.139199 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 17:27:59.139199 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:27:59.139199 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 17:27:59.139199 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:27:59.139199 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:27:59.139199 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:27:59.139199 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:27:59.139199 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:27:59.139199 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:27:59.139199 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application with Lock( Aug 24 17:27:59.139199 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:27:59.139199 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:27:59.139199 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:27:59.139199 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:27:59.139199 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:27:59.139199 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:27:59.140105 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:27:59.140105 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:27:59.140105 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 17:27:59.140105 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 17:27:59.140105 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 17:27:59.140105 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 17:27:59.140105 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 17:27:59.140105 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 17:27:59.140105 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 17:27:59.140105 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Aug 24 17:27:59.146116 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 290/571] 10.222.0.45 () {56 vars in 1103 bytes} [Wed Aug 24 17:27:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:27:59.146116 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa653d3a-def3-4625-8d46-8dd90047214a None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:59.146116 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa653d3a-def3-4625-8d46-8dd90047214a None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:59.146116 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa653d3a-def3-4625-8d46-8dd90047214a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:59.146435 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 282/572] 10.222.0.45 () {56 vars in 1093 bytes} [Wed Aug 24 17:27:58 2022] GET /identity/v3/users/nova => generated 82 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:27:59.146435 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9d6d7b-e540-4f43-8c5e-fc0ce0a1135c None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:59.161491 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-aa653d3a-def3-4625-8d46-8dd90047214a None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:27:59.162666 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-aa653d3a-def3-4625-8d46-8dd90047214a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:27:59.217518 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:59.229918 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 291/573] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:27:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:27:59.230837 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa653d3a-def3-4625-8d46-8dd90047214a None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:59.255580 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9d6d7b-e540-4f43-8c5e-fc0ce0a1135c None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:59.270682 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:59.292812 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b31d744-876d-4b75-8707-bb3036ad3db5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:59.293412 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b31d744-876d-4b75-8707-bb3036ad3db5 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:59.293694 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b31d744-876d-4b75-8707-bb3036ad3db5 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:59.293962 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b31d744-876d-4b75-8707-bb3036ad3db5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:59.294676 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b31d744-876d-4b75-8707-bb3036ad3db5 None admin] RBAC: Adding query filter params (role.id=ad6a81fb5d1941888577eb0ce9474882, scope.project.id=2ad8a32bd9d2434097bd08b111914ca1, user.id=90f0772b64c84faaa689e7272fa97898) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:27:59.295087 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b31d744-876d-4b75-8707-bb3036ad3db5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:59.296646 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b31d744-876d-4b75-8707-bb3036ad3db5 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:59.312670 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 283/574] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Aug 24 17:27:59 2022] GET /identity/v3/role_assignments?user.id=90f0772b64c84faaa689e7272fa97898&scope.project.id=2ad8a32bd9d2434097bd08b111914ca1&role.id=ad6a81fb5d1941888577eb0ce9474882 => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:27:59.312670 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b31d744-876d-4b75-8707-bb3036ad3db5 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:59.345089 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa653d3a-def3-4625-8d46-8dd90047214a None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:59.390227 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c36ef6-7291-4fd3-b904-866d64740dbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:59.390974 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c36ef6-7291-4fd3-b904-866d64740dbd None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:59.391322 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c36ef6-7291-4fd3-b904-866d64740dbd None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:59.391659 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c36ef6-7291-4fd3-b904-866d64740dbd None admin] PATH_INFO: `/v3/users/glance` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:59.400157 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c36ef6-7291-4fd3-b904-866d64740dbd None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:59.402431 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c36ef6-7291-4fd3-b904-866d64740dbd None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:59.415487 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application [None req-b1c36ef6-7291-4fd3-b904-866d64740dbd None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 24 17:27:59.415487 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:27:59.415487 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:27:59.415487 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:27:59.415487 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:27:59.415487 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:27:59.415487 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:27:59.415487 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:27:59.415487 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:27:59.415487 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:27:59.415487 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:27:59.415487 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:27:59.415487 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 17:27:59.415487 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 17:27:59.415487 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 17:27:59.415487 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 17:27:59.417356 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:27:59.417356 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:27:59.417356 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 17:27:59.417356 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:27:59.417356 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 17:27:59.417356 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:27:59.417356 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:27:59.417356 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:27:59.417356 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:27:59.417356 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:27:59.417356 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:27:59.417356 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application with Lock( Aug 24 17:27:59.417356 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:27:59.417356 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:27:59.417356 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:27:59.417356 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:27:59.417356 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:27:59.417356 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:27:59.418504 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:27:59.418504 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:27:59.418504 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 17:27:59.418504 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 17:27:59.418504 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 17:27:59.418504 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 17:27:59.418504 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 17:27:59.418504 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 17:27:59.418504 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 24 17:27:59.418504 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Aug 24 17:27:59.424751 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 292/575] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Aug 24 17:27:59 2022] GET /identity/v3/users/glance => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:27:59.425813 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c36ef6-7291-4fd3-b904-866d64740dbd None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:59.442811 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b31d744-876d-4b75-8707-bb3036ad3db5 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:59.480005 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656e1a11-6cd6-4a96-a17a-c440626dc6c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:59.480844 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656e1a11-6cd6-4a96-a17a-c440626dc6c9 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:59.481124 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656e1a11-6cd6-4a96-a17a-c440626dc6c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:59.481397 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656e1a11-6cd6-4a96-a17a-c440626dc6c9 None admin] PATH_INFO: `/v3/users` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:59.482439 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656e1a11-6cd6-4a96-a17a-c440626dc6c9 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:27:59.482744 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656e1a11-6cd6-4a96-a17a-c440626dc6c9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:59.484197 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656e1a11-6cd6-4a96-a17a-c440626dc6c9 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:59.500716 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:59.528886 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 284/576] 10.222.0.45 () {56 vars in 1133 bytes} [Wed Aug 24 17:27:59 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:27:59.528886 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656e1a11-6cd6-4a96-a17a-c440626dc6c9 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:59.534232 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:59.537460 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c36ef6-7291-4fd3-b904-866d64740dbd None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:59.571660 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f342cc8e-2751-4d43-bb5a-05881d1f01d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:59.572214 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f342cc8e-2751-4d43-bb5a-05881d1f01d3 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:59.573040 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f342cc8e-2751-4d43-bb5a-05881d1f01d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:59.573335 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f342cc8e-2751-4d43-bb5a-05881d1f01d3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:59.584708 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f342cc8e-2751-4d43-bb5a-05881d1f01d3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:59.586174 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f342cc8e-2751-4d43-bb5a-05881d1f01d3 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:59.601170 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application [None req-f342cc8e-2751-4d43-bb5a-05881d1f01d3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:27:59.601170 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:27:59.601170 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:27:59.601170 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:27:59.601170 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:27:59.601170 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:27:59.601170 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:27:59.601170 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:27:59.601170 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:27:59.601170 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:27:59.601170 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:27:59.601170 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:27:59.601170 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:27:59.601170 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:27:59.601170 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:27:59.601170 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:27:59.602347 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:27:59.602347 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:27:59.602347 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:27:59.602347 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:27:59.602347 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:27:59.602347 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:27:59.602347 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:27:59.602347 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application with Lock( Aug 24 17:27:59.602347 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:27:59.602347 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:27:59.602347 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:27:59.602347 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:27:59.602347 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:27:59.602347 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:27:59.602347 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:27:59.602347 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:27:59.602347 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:27:59.603405 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:27:59.603405 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:27:59.603405 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Aug 24 17:27:59.607191 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 293/577] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 17:27:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:27:59.608025 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f342cc8e-2751-4d43-bb5a-05881d1f01d3 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:59.640152 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656e1a11-6cd6-4a96-a17a-c440626dc6c9 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:59.677991 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b40642b-8814-4e20-85ea-950a9ad0e6c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:59.678834 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b40642b-8814-4e20-85ea-950a9ad0e6c0 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:59.679270 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b40642b-8814-4e20-85ea-950a9ad0e6c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:59.679553 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b40642b-8814-4e20-85ea-950a9ad0e6c0 None admin] PATH_INFO: `/v3/users` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:59.680282 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b40642b-8814-4e20-85ea-950a9ad0e6c0 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:27:59.680613 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b40642b-8814-4e20-85ea-950a9ad0e6c0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:59.682578 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b40642b-8814-4e20-85ea-950a9ad0e6c0 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:59.700969 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 285/578] 10.222.0.45 () {56 vars in 1137 bytes} [Wed Aug 24 17:27:59 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:27:59.701791 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b40642b-8814-4e20-85ea-950a9ad0e6c0 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:59.714793 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f342cc8e-2751-4d43-bb5a-05881d1f01d3 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:59.749311 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2961d48-ecc7-4656-9059-b777b29509d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:59.750041 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2961d48-ecc7-4656-9059-b777b29509d3 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:59.750325 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2961d48-ecc7-4656-9059-b777b29509d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:59.750604 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2961d48-ecc7-4656-9059-b777b29509d3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:59.761494 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:59.761917 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2961d48-ecc7-4656-9059-b777b29509d3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:27:59.767175 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application [None req-a2961d48-ecc7-4656-9059-b777b29509d3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 17:27:59.767175 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:27:59.767175 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:27:59.767175 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:27:59.767175 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:27:59.767175 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:27:59.767175 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:27:59.767175 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:27:59.767175 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:27:59.767175 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:27:59.767175 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:27:59.767175 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:27:59.767175 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:27:59.767175 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:27:59.767175 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:27:59.767175 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:27:59.768099 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:27:59.768099 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:27:59.768099 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:27:59.768099 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:27:59.768099 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:27:59.768099 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:27:59.768099 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:27:59.768099 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:27:59.768099 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:27:59.768099 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:27:59.768099 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:27:59.768099 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application with Lock( Aug 24 17:27:59.768099 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:27:59.768099 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:27:59.768099 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:27:59.768099 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:27:59.768982 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:27:59.768982 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:27:59.768982 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:27:59.768982 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:27:59.768982 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:27:59.768982 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:27:59.768982 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:27:59.768982 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:27:59.768982 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:27:59.768982 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:27:59.768982 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 17:27:59.768982 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Aug 24 17:27:59.768982 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 294/579] 10.222.0.45 () {56 vars in 1102 bytes} [Wed Aug 24 17:27:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:27:59.769693 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2961d48-ecc7-4656-9059-b777b29509d3 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:59.786377 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:59.821435 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b40642b-8814-4e20-85ea-950a9ad0e6c0 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:59.862463 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a68388-b203-45f4-b5c9-4e79126c1237 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:59.863116 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a68388-b203-45f4-b5c9-4e79126c1237 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:59.863469 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a68388-b203-45f4-b5c9-4e79126c1237 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:59.863809 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a68388-b203-45f4-b5c9-4e79126c1237 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:59.874619 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a68388-b203-45f4-b5c9-4e79126c1237 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:59.876627 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a68388-b203-45f4-b5c9-4e79126c1237 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:59.886908 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2961d48-ecc7-4656-9059-b777b29509d3 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:59.891635 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application [None req-83a68388-b203-45f4-b5c9-4e79126c1237 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:27:59.891635 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:27:59.891635 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:27:59.891635 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:27:59.891635 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:27:59.891635 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:27:59.891635 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:27:59.891635 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:27:59.891635 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:27:59.891635 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:27:59.891635 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:27:59.891635 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:27:59.891635 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:27:59.891635 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:27:59.891635 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:27:59.891635 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:27:59.892683 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:27:59.892683 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:27:59.892683 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:27:59.892683 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:27:59.892683 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:27:59.892683 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:27:59.892683 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:27:59.892683 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application with Lock( Aug 24 17:27:59.892683 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:27:59.892683 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:27:59.892683 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:27:59.892683 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:27:59.892683 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:27:59.892683 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:27:59.892683 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:27:59.892683 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:27:59.892683 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:27:59.898935 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:27:59.898935 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:27:59.898935 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Aug 24 17:27:59.898935 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 286/580] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 17:27:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:27:59.898935 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a68388-b203-45f4-b5c9-4e79126c1237 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:59.925538 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6782b7-863b-4bb4-a7c8-49078258d3a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:59.926144 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6782b7-863b-4bb4-a7c8-49078258d3a5 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:59.926440 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6782b7-863b-4bb4-a7c8-49078258d3a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:59.926695 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6782b7-863b-4bb4-a7c8-49078258d3a5 None admin] PATH_INFO: `/v3/domains` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:59.927333 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6782b7-863b-4bb4-a7c8-49078258d3a5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:27:59.927641 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6782b7-863b-4bb4-a7c8-49078258d3a5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:59.929056 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6782b7-863b-4bb4-a7c8-49078258d3a5 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:59.945639 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 295/581] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 24 17:27:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:27:59.945639 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6782b7-863b-4bb4-a7c8-49078258d3a5 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:00.014432 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:00.015102 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a68388-b203-45f4-b5c9-4e79126c1237 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:00.018397 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:00.045570 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6782b7-863b-4bb4-a7c8-49078258d3a5 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:00.053848 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013ad8ca-5134-4979-a3a9-388731f56f5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:00.054476 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013ad8ca-5134-4979-a3a9-388731f56f5c None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:00.054760 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013ad8ca-5134-4979-a3a9-388731f56f5c None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:00.055030 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013ad8ca-5134-4979-a3a9-388731f56f5c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:00.064848 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013ad8ca-5134-4979-a3a9-388731f56f5c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:00.066309 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013ad8ca-5134-4979-a3a9-388731f56f5c None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:00.081768 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a0d2ff-1d14-4451-acee-ae5d009a6159 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:00.082446 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a0d2ff-1d14-4451-acee-ae5d009a6159 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:00.082714 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application [None req-013ad8ca-5134-4979-a3a9-388731f56f5c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:28:00.082714 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:28:00.082714 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:28:00.082714 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:28:00.082714 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:28:00.082714 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:28:00.082714 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:28:00.082714 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:28:00.082714 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:28:00.082714 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:28:00.082714 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:28:00.082714 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:28:00.082714 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:28:00.082714 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:28:00.082714 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:28:00.082714 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:28:00.083792 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:28:00.083792 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:28:00.083792 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:28:00.083792 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:28:00.083792 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:28:00.083792 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:28:00.083792 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:28:00.083792 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application with Lock( Aug 24 17:28:00.083792 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:28:00.083792 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:28:00.083792 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:28:00.083792 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:28:00.083792 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:28:00.083792 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:28:00.083792 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:28:00.083792 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:28:00.083792 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:28:00.085003 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:28:00.085003 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:28:00.085003 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Aug 24 17:28:00.085192 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a0d2ff-1d14-4451-acee-ae5d009a6159 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:00.085192 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a0d2ff-1d14-4451-acee-ae5d009a6159 None admin] PATH_INFO: `/v3/roles` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:00.085192 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a0d2ff-1d14-4451-acee-ae5d009a6159 None admin] RBAC: Adding query filter params (name=service) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:00.085192 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a0d2ff-1d14-4451-acee-ae5d009a6159 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:00.086526 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a0d2ff-1d14-4451-acee-ae5d009a6159 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:00.089250 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 287/582] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 17:27:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:28:00.090107 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013ad8ca-5134-4979-a3a9-388731f56f5c None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:00.100958 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 296/583] 10.222.0.45 () {56 vars in 1103 bytes} [Wed Aug 24 17:27:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:00.100958 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a0d2ff-1d14-4451-acee-ae5d009a6159 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:00.199079 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013ad8ca-5134-4979-a3a9-388731f56f5c None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:00.205752 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a0d2ff-1d14-4451-acee-ae5d009a6159 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:00.235946 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895be563-1a30-4fc9-ab49-bd4def6058b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:00.236592 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895be563-1a30-4fc9-ab49-bd4def6058b5 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:00.236861 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895be563-1a30-4fc9-ab49-bd4def6058b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:00.237137 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895be563-1a30-4fc9-ab49-bd4def6058b5 None admin] PATH_INFO: `/v3/domains` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:00.237781 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895be563-1a30-4fc9-ab49-bd4def6058b5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:00.238085 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895be563-1a30-4fc9-ab49-bd4def6058b5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:00.239453 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895be563-1a30-4fc9-ab49-bd4def6058b5 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:00.240317 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342b85e3-f587-4f86-8b64-30287fe7ef55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:00.240974 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342b85e3-f587-4f86-8b64-30287fe7ef55 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:00.241254 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342b85e3-f587-4f86-8b64-30287fe7ef55 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:00.241525 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342b85e3-f587-4f86-8b64-30287fe7ef55 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:00.243481 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:00.251491 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342b85e3-f587-4f86-8b64-30287fe7ef55 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:00.256189 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 288/584] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 24 17:28:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:00.256189 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895be563-1a30-4fc9-ab49-bd4def6058b5 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:00.256411 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342b85e3-f587-4f86-8b64-30287fe7ef55 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:00.268306 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application [None req-342b85e3-f587-4f86-8b64-30287fe7ef55 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 17:28:00.268306 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:28:00.268306 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:28:00.268306 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:28:00.268306 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:28:00.268306 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:28:00.268306 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:28:00.268306 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:28:00.268306 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:28:00.268306 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:28:00.268306 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:28:00.268306 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:28:00.268306 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 17:28:00.268306 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 17:28:00.268306 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 17:28:00.268306 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 17:28:00.274877 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:28:00.274877 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:28:00.274877 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 17:28:00.274877 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:28:00.274877 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 17:28:00.274877 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:28:00.274877 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:28:00.274877 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:28:00.274877 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:28:00.274877 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:28:00.274877 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:28:00.274877 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application with Lock( Aug 24 17:28:00.274877 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:28:00.274877 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:28:00.274877 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:28:00.274877 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:28:00.274877 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:28:00.274877 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:28:00.276597 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:28:00.276597 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:28:00.276597 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 17:28:00.276597 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 17:28:00.276597 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 17:28:00.276597 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 17:28:00.276597 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 17:28:00.276597 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 17:28:00.276597 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 17:28:00.276597 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Aug 24 17:28:00.276597 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 297/585] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Aug 24 17:28:00 2022] GET /identity/v3/users/cinder => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:28:00.276597 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342b85e3-f587-4f86-8b64-30287fe7ef55 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:00.284484 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:00.373732 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895be563-1a30-4fc9-ab49-bd4def6058b5 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:00.380926 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342b85e3-f587-4f86-8b64-30287fe7ef55 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:00.410347 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405e4c4f-cb4c-4bdd-971b-9b077221797d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:00.410951 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405e4c4f-cb4c-4bdd-971b-9b077221797d None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:00.411229 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405e4c4f-cb4c-4bdd-971b-9b077221797d None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:00.411523 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405e4c4f-cb4c-4bdd-971b-9b077221797d None admin] PATH_INFO: `/v3/domains` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:00.412125 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405e4c4f-cb4c-4bdd-971b-9b077221797d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:00.412431 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405e4c4f-cb4c-4bdd-971b-9b077221797d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:00.413818 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405e4c4f-cb4c-4bdd-971b-9b077221797d None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:00.417354 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf15ef2a-7f12-4a86-96c3-8e6ed81f7c7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:00.417987 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf15ef2a-7f12-4a86-96c3-8e6ed81f7c7c None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:00.418270 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf15ef2a-7f12-4a86-96c3-8e6ed81f7c7c None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:00.418547 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf15ef2a-7f12-4a86-96c3-8e6ed81f7c7c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:00.429528 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 289/586] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 24 17:28:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:00.429528 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405e4c4f-cb4c-4bdd-971b-9b077221797d None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:00.429781 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf15ef2a-7f12-4a86-96c3-8e6ed81f7c7c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:00.430660 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf15ef2a-7f12-4a86-96c3-8e6ed81f7c7c None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:00.446456 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application [None req-cf15ef2a-7f12-4a86-96c3-8e6ed81f7c7c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:28:00.446456 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:28:00.446456 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:28:00.446456 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:28:00.446456 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:28:00.446456 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:28:00.446456 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:28:00.446456 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:28:00.446456 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:28:00.446456 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:28:00.446456 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:28:00.446456 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:28:00.446456 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:28:00.446456 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:28:00.446456 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:28:00.446456 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:28:00.447651 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:28:00.447651 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:28:00.447651 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:28:00.447651 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:28:00.447651 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:28:00.447651 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:28:00.447651 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:28:00.447651 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application with Lock( Aug 24 17:28:00.447651 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:28:00.447651 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:28:00.447651 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:28:00.447651 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:28:00.447651 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:28:00.447651 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:28:00.447651 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:28:00.447651 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:28:00.447651 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:28:00.449308 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:28:00.449308 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:28:00.449308 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Aug 24 17:28:00.449725 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 298/587] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 17:28:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:28:00.450494 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf15ef2a-7f12-4a86-96c3-8e6ed81f7c7c None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:00.497648 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:00.519745 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:00.544164 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405e4c4f-cb4c-4bdd-971b-9b077221797d None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:00.551588 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf15ef2a-7f12-4a86-96c3-8e6ed81f7c7c None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:00.579985 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d09915-2f22-4a13-b68e-75dc815d2204 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:00.580719 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d09915-2f22-4a13-b68e-75dc815d2204 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:00.581065 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d09915-2f22-4a13-b68e-75dc815d2204 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:00.581408 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d09915-2f22-4a13-b68e-75dc815d2204 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:00.584108 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fe1205-769a-4f8b-bf72-cd352cc48698 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:00.584737 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fe1205-769a-4f8b-bf72-cd352cc48698 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:00.585019 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fe1205-769a-4f8b-bf72-cd352cc48698 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:00.585292 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fe1205-769a-4f8b-bf72-cd352cc48698 None admin] PATH_INFO: `/v3/users` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:00.586019 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fe1205-769a-4f8b-bf72-cd352cc48698 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:00.586322 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fe1205-769a-4f8b-bf72-cd352cc48698 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:00.587739 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fe1205-769a-4f8b-bf72-cd352cc48698 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:00.589104 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d09915-2f22-4a13-b68e-75dc815d2204 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:00.590843 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d09915-2f22-4a13-b68e-75dc815d2204 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:00.603520 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application [None req-b6d09915-2f22-4a13-b68e-75dc815d2204 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:28:00.603520 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:28:00.603520 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:28:00.603520 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:28:00.603520 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:28:00.603520 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:28:00.603520 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:28:00.603520 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:28:00.603520 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:28:00.603520 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:28:00.603520 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:28:00.603520 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:28:00.603520 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 17:28:00.603520 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 17:28:00.603520 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 17:28:00.604764 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 17:28:00.604764 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:28:00.604764 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:28:00.604764 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:28:00.604764 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:28:00.604764 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:28:00.604764 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:28:00.604764 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:28:00.604764 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application with Lock( Aug 24 17:28:00.604764 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:28:00.604764 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:28:00.604764 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:28:00.604764 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:28:00.604764 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:28:00.604764 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:28:00.604764 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:28:00.604764 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:28:00.605754 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 17:28:00.605754 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 17:28:00.605754 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 17:28:00.605754 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 17:28:00.605754 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 17:28:00.605754 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 17:28:00.605754 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:28:00.605754 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Aug 24 17:28:00.611946 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 299/588] 10.222.0.45 () {56 vars in 1137 bytes} [Wed Aug 24 17:28:00 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:00.611946 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fe1205-769a-4f8b-bf72-cd352cc48698 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:00.612160 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 290/589] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Aug 24 17:28:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:28:00.612160 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d09915-2f22-4a13-b68e-75dc815d2204 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:00.716317 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d09915-2f22-4a13-b68e-75dc815d2204 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:00.718543 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fe1205-769a-4f8b-bf72-cd352cc48698 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:00.721811 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:00.752468 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1108b1c-ee5d-47b2-bfd2-b914caf3bac5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:00.753108 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1108b1c-ee5d-47b2-bfd2-b914caf3bac5 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:00.753393 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1108b1c-ee5d-47b2-bfd2-b914caf3bac5 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:00.753664 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1108b1c-ee5d-47b2-bfd2-b914caf3bac5 None admin] PATH_INFO: `/v3/domains` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:00.754315 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1108b1c-ee5d-47b2-bfd2-b914caf3bac5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:00.754609 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1108b1c-ee5d-47b2-bfd2-b914caf3bac5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:00.755844 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10799b71-a1ab-4053-ae46-4a7ff9542e65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:00.755985 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1108b1c-ee5d-47b2-bfd2-b914caf3bac5 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:00.756770 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10799b71-a1ab-4053-ae46-4a7ff9542e65 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:00.757192 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10799b71-a1ab-4053-ae46-4a7ff9542e65 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:00.757598 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10799b71-a1ab-4053-ae46-4a7ff9542e65 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:00.758962 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:00.765888 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10799b71-a1ab-4053-ae46-4a7ff9542e65 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:00.767482 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10799b71-a1ab-4053-ae46-4a7ff9542e65 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:00.770820 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 291/590] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 24 17:28:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:00.771823 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1108b1c-ee5d-47b2-bfd2-b914caf3bac5 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:00.783493 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application [None req-10799b71-a1ab-4053-ae46-4a7ff9542e65 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:28:00.783493 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:28:00.783493 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:28:00.783493 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:28:00.783493 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:28:00.783493 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:28:00.783493 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:28:00.783493 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:28:00.783493 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:28:00.783493 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:28:00.783493 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:28:00.783493 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:28:00.783493 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 17:28:00.783493 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 17:28:00.783493 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 17:28:00.784413 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 17:28:00.784413 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:28:00.784413 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:28:00.784413 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:28:00.784413 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:28:00.784413 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:28:00.784413 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:28:00.784413 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:28:00.784413 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application with Lock( Aug 24 17:28:00.784413 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:28:00.784413 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:28:00.784413 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:28:00.784413 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:28:00.784413 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:28:00.784413 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:28:00.784413 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:28:00.784413 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:28:00.785341 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 17:28:00.785341 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 17:28:00.785341 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 17:28:00.785341 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 17:28:00.785341 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 17:28:00.785341 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 17:28:00.785341 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:28:00.785341 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Aug 24 17:28:00.785341 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 300/591] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Aug 24 17:28:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:28:00.786711 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10799b71-a1ab-4053-ae46-4a7ff9542e65 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:00.883136 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1108b1c-ee5d-47b2-bfd2-b914caf3bac5 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:00.898905 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10799b71-a1ab-4053-ae46-4a7ff9542e65 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:00.919214 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4a845e-456f-433c-a61d-e87224fd9563 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:00.919948 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4a845e-456f-433c-a61d-e87224fd9563 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:00.920352 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4a845e-456f-433c-a61d-e87224fd9563 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:00.920765 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4a845e-456f-433c-a61d-e87224fd9563 None admin] PATH_INFO: `/v3/projects` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:00.921562 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4a845e-456f-433c-a61d-e87224fd9563 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:00.921976 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4a845e-456f-433c-a61d-e87224fd9563 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:00.923514 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4a845e-456f-433c-a61d-e87224fd9563 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:00.935572 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 292/592] 10.222.0.45 () {56 vars in 1148 bytes} [Wed Aug 24 17:28:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:00.936438 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4a845e-456f-433c-a61d-e87224fd9563 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:00.941803 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c3bcbf-078a-4d7f-9e47-74c1f2778fd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:00.942618 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c3bcbf-078a-4d7f-9e47-74c1f2778fd0 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:00.943044 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c3bcbf-078a-4d7f-9e47-74c1f2778fd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:00.943548 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c3bcbf-078a-4d7f-9e47-74c1f2778fd0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:00.954729 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c3bcbf-078a-4d7f-9e47-74c1f2778fd0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:00.956688 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c3bcbf-078a-4d7f-9e47-74c1f2778fd0 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:00.972511 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application [None req-86c3bcbf-078a-4d7f-9e47-74c1f2778fd0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:28:00.972511 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:28:00.972511 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:28:00.972511 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:28:00.972511 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:28:00.972511 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:28:00.972511 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:28:00.972511 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:28:00.972511 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:28:00.972511 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:28:00.972511 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:28:00.972511 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:28:00.972511 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:28:00.972511 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:28:00.972511 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:28:00.972511 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:28:00.973635 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:28:00.973635 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:28:00.973635 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:28:00.973635 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:28:00.973635 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:28:00.973635 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:28:00.973635 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:28:00.973635 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application with Lock( Aug 24 17:28:00.973635 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:28:00.973635 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:28:00.973635 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:28:00.973635 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:28:00.973635 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:28:00.973635 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:28:00.973635 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:28:00.973635 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:28:00.973635 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:28:00.974704 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:28:00.974704 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:28:00.974704 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Aug 24 17:28:00.981993 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 301/593] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 17:28:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:28:00.982998 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c3bcbf-078a-4d7f-9e47-74c1f2778fd0 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:00.993687 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:01.002448 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:01.043630 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4a845e-456f-433c-a61d-e87224fd9563 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:01.078442 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4af3bc6-8fbd-4184-9cbb-ef3046e6b212 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:01.079164 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4af3bc6-8fbd-4184-9cbb-ef3046e6b212 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:01.079513 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4af3bc6-8fbd-4184-9cbb-ef3046e6b212 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:01.079845 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4af3bc6-8fbd-4184-9cbb-ef3046e6b212 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:01.087595 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4af3bc6-8fbd-4184-9cbb-ef3046e6b212 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:01.089079 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4af3bc6-8fbd-4184-9cbb-ef3046e6b212 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:01.097021 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c3bcbf-078a-4d7f-9e47-74c1f2778fd0 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:01.100216 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application [None req-d4af3bc6-8fbd-4184-9cbb-ef3046e6b212 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 17:28:01.100216 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:28:01.100216 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:28:01.100216 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:28:01.100216 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:28:01.100216 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:28:01.100216 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:28:01.100216 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:28:01.100216 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:28:01.100216 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:28:01.100216 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:28:01.100216 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:28:01.100216 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 17:28:01.100216 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 17:28:01.100216 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 17:28:01.100216 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 17:28:01.101237 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:28:01.101237 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:28:01.101237 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 17:28:01.101237 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:28:01.101237 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 17:28:01.101237 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:28:01.101237 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:28:01.101237 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:28:01.101237 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:28:01.101237 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:28:01.101237 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:28:01.101237 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application with Lock( Aug 24 17:28:01.101237 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:28:01.101237 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:28:01.101237 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:28:01.101237 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:28:01.101237 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:28:01.101237 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:28:01.107737 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:28:01.107737 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:28:01.107737 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 17:28:01.107737 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 17:28:01.107737 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 17:28:01.107737 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 17:28:01.107737 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 17:28:01.107737 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 17:28:01.107737 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 17:28:01.107737 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Aug 24 17:28:01.107737 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 293/594] 10.222.0.45 () {56 vars in 1102 bytes} [Wed Aug 24 17:28:00 2022] GET /identity/v3/users/neutron => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:28:01.107737 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4af3bc6-8fbd-4184-9cbb-ef3046e6b212 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:01.138569 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ee282b-a8da-4324-be98-a2810a783bb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:01.139182 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ee282b-a8da-4324-be98-a2810a783bb5 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:01.139463 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ee282b-a8da-4324-be98-a2810a783bb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:01.139739 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ee282b-a8da-4324-be98-a2810a783bb5 None admin] PATH_INFO: `/v3/projects` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:01.140464 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ee282b-a8da-4324-be98-a2810a783bb5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:01.140819 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ee282b-a8da-4324-be98-a2810a783bb5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:01.142255 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ee282b-a8da-4324-be98-a2810a783bb5 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:01.160138 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 302/595] 10.222.0.45 () {56 vars in 1148 bytes} [Wed Aug 24 17:28:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:01.160138 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ee282b-a8da-4324-be98-a2810a783bb5 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:01.217283 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4af3bc6-8fbd-4184-9cbb-ef3046e6b212 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:01.234827 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:01.241061 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:01.253314 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f21b3b-1065-4774-9048-1d911377504b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:01.253969 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f21b3b-1065-4774-9048-1d911377504b None admin] REQUEST_METHOD: `PUT` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:01.254246 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f21b3b-1065-4774-9048-1d911377504b None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:01.254513 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f21b3b-1065-4774-9048-1d911377504b None admin] PATH_INFO: `/v3/projects/2ad8a32bd9d2434097bd08b111914ca1/users/053b7ce6d6f649ed91a26e434dab805e/roles/ad6a81fb5d1941888577eb0ce9474882` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:01.263561 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ee282b-a8da-4324-be98-a2810a783bb5 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:01.268571 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f21b3b-1065-4774-9048-1d911377504b None admin] RBAC: Authorizing `identity:create_grant(project_id=2ad8a32bd9d2434097bd08b111914ca1, user_id=053b7ce6d6f649ed91a26e434dab805e, role_id=ad6a81fb5d1941888577eb0ce9474882)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:01.270021 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f21b3b-1065-4774-9048-1d911377504b None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:01.283760 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 294/596] 10.222.0.45 () {58 vars in 1439 bytes} [Wed Aug 24 17:28:01 2022] PUT /identity/v3/projects/2ad8a32bd9d2434097bd08b111914ca1/users/053b7ce6d6f649ed91a26e434dab805e/roles/ad6a81fb5d1941888577eb0ce9474882 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:28:01.284573 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f21b3b-1065-4774-9048-1d911377504b None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:01.300250 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf4d3d9-7146-4ebe-af72-57a02037a0ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:01.300918 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf4d3d9-7146-4ebe-af72-57a02037a0ce None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:01.301207 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf4d3d9-7146-4ebe-af72-57a02037a0ce None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:01.301685 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf4d3d9-7146-4ebe-af72-57a02037a0ce None admin] PATH_INFO: `/v3/domains` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:01.302422 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf4d3d9-7146-4ebe-af72-57a02037a0ce None admin] RBAC: Adding query filter params (name=Default) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:01.302803 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf4d3d9-7146-4ebe-af72-57a02037a0ce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:01.304299 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf4d3d9-7146-4ebe-af72-57a02037a0ce None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:01.317051 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 303/597] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 24 17:28:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:01.322461 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf4d3d9-7146-4ebe-af72-57a02037a0ce None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:01.393712 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f21b3b-1065-4774-9048-1d911377504b None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:01.438399 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf4d3d9-7146-4ebe-af72-57a02037a0ce None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:01.443759 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b7b043-e98f-426c-a5c9-d3cd9a46111a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:01.443759 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b7b043-e98f-426c-a5c9-d3cd9a46111a None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:01.443759 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b7b043-e98f-426c-a5c9-d3cd9a46111a None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:01.443759 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b7b043-e98f-426c-a5c9-d3cd9a46111a None admin] PATH_INFO: `/v3/users` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:01.444155 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b7b043-e98f-426c-a5c9-d3cd9a46111a None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:01.444155 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b7b043-e98f-426c-a5c9-d3cd9a46111a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:01.445604 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b7b043-e98f-426c-a5c9-d3cd9a46111a None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:01.463999 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 295/598] 10.222.0.45 () {56 vars in 1139 bytes} [Wed Aug 24 17:28:01 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:01.465412 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1515ead-f7ac-460a-a82b-a80c1f7c6c25 None None] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:01.465774 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1515ead-f7ac-460a-a82b-a80c1f7c6c25 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:01.466111 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1515ead-f7ac-460a-a82b-a80c1f7c6c25 None None] PATH_INFO: `/` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:01.467503 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 296/599] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:28:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:28:01.468130 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1515ead-f7ac-460a-a82b-a80c1f7c6c25 None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:01.477865 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80efe5d8-55df-4c29-bb68-a1120bb97c61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:01.478524 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80efe5d8-55df-4c29-bb68-a1120bb97c61 None admin] REQUEST_METHOD: `PUT` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:01.478817 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80efe5d8-55df-4c29-bb68-a1120bb97c61 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:01.479851 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80efe5d8-55df-4c29-bb68-a1120bb97c61 None admin] PATH_INFO: `/v3/projects/2ad8a32bd9d2434097bd08b111914ca1/users/df5fee57539447428dd1c72eab3cfd9d/roles/ad6a81fb5d1941888577eb0ce9474882` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:01.484800 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:01.496019 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:01.497745 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80efe5d8-55df-4c29-bb68-a1120bb97c61 None admin] RBAC: Authorizing `identity:create_grant(project_id=2ad8a32bd9d2434097bd08b111914ca1, user_id=df5fee57539447428dd1c72eab3cfd9d, role_id=ad6a81fb5d1941888577eb0ce9474882)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:01.499199 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80efe5d8-55df-4c29-bb68-a1120bb97c61 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:01.520581 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 304/600] 10.222.0.45 () {58 vars in 1439 bytes} [Wed Aug 24 17:28:01 2022] PUT /identity/v3/projects/2ad8a32bd9d2434097bd08b111914ca1/users/df5fee57539447428dd1c72eab3cfd9d/roles/ad6a81fb5d1941888577eb0ce9474882 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:28:01.520581 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80efe5d8-55df-4c29-bb68-a1120bb97c61 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:01.592275 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1515ead-f7ac-460a-a82b-a80c1f7c6c25 None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:01.641068 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80efe5d8-55df-4c29-bb68-a1120bb97c61 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:01.643424 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba5b759-5823-4569-b5ea-b4f3a409bcfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:01.645121 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba5b759-5823-4569-b5ea-b4f3a409bcfd None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:01.645481 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba5b759-5823-4569-b5ea-b4f3a409bcfd None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:01.645832 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba5b759-5823-4569-b5ea-b4f3a409bcfd None admin] PATH_INFO: `/v3/projects/service` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:01.656406 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba5b759-5823-4569-b5ea-b4f3a409bcfd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:01.658707 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba5b759-5823-4569-b5ea-b4f3a409bcfd None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:01.677360 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application [None req-7ba5b759-5823-4569-b5ea-b4f3a409bcfd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:28:01.677360 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:28:01.677360 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:28:01.677360 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:28:01.677360 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:28:01.677360 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:28:01.677360 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:28:01.677360 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:28:01.677360 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:28:01.677360 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:28:01.677360 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:28:01.677360 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:28:01.677360 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 17:28:01.677360 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 17:28:01.677360 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 17:28:01.678549 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 17:28:01.678549 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:28:01.678549 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:28:01.678549 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:28:01.678549 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:28:01.678549 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:28:01.678549 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:28:01.678549 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:28:01.678549 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application with Lock( Aug 24 17:28:01.678549 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:28:01.678549 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:28:01.678549 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:28:01.678549 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:28:01.678549 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:28:01.678549 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:28:01.678549 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:28:01.678549 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:28:01.679631 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 17:28:01.679631 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 17:28:01.679631 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 17:28:01.679631 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 17:28:01.679631 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 17:28:01.679631 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 17:28:01.679631 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:28:01.679631 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Aug 24 17:28:01.681119 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 297/601] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Aug 24 17:28:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:28:01.682545 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e51f3e5-085d-4c63-8751-d206dca51147 None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:01.682900 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e51f3e5-085d-4c63-8751-d206dca51147 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:01.683277 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e51f3e5-085d-4c63-8751-d206dca51147 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:01.693375 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cc469c-a748-42b3-98c8-c4a39120cc91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:01.694236 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cc469c-a748-42b3-98c8-c4a39120cc91 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:01.694692 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cc469c-a748-42b3-98c8-c4a39120cc91 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:01.695127 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cc469c-a748-42b3-98c8-c4a39120cc91 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:01.703255 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-7e51f3e5-085d-4c63-8751-d206dca51147 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:28:01.704474 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-7e51f3e5-085d-4c63-8751-d206dca51147 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:28:01.708438 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cc469c-a748-42b3-98c8-c4a39120cc91 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:01.710067 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cc469c-a748-42b3-98c8-c4a39120cc91 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:01.727105 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application [None req-59cc469c-a748-42b3-98c8-c4a39120cc91 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:28:01.727105 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:28:01.727105 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:28:01.727105 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:28:01.727105 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:28:01.727105 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:28:01.727105 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:28:01.727105 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:28:01.727105 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:28:01.727105 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:28:01.727105 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:28:01.727105 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:28:01.727105 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:28:01.727105 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:28:01.727105 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:28:01.727105 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:28:01.728259 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:28:01.728259 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:28:01.728259 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:28:01.728259 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:28:01.728259 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:28:01.728259 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:28:01.728259 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:28:01.728259 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application with Lock( Aug 24 17:28:01.728259 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:28:01.728259 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:28:01.728259 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:28:01.728259 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:28:01.728259 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:28:01.728259 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:28:01.728259 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:28:01.728259 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:28:01.728259 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:28:01.729251 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:28:01.729251 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:28:01.729251 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Aug 24 17:28:01.729251 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 305/602] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 17:28:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:28:01.729251 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cc469c-a748-42b3-98c8-c4a39120cc91 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:01.790521 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:01.798432 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 298/603] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:28:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:28:01.799299 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e51f3e5-085d-4c63-8751-d206dca51147 None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:01.828276 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:01.861756 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cc469c-a748-42b3-98c8-c4a39120cc91 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:01.922022 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948f7578-1e2d-4db0-a286-3e73fe4d6bb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:01.923193 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948f7578-1e2d-4db0-a286-3e73fe4d6bb6 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:01.923931 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948f7578-1e2d-4db0-a286-3e73fe4d6bb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:01.924360 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948f7578-1e2d-4db0-a286-3e73fe4d6bb6 None admin] PATH_INFO: `/v3/projects` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:01.925356 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948f7578-1e2d-4db0-a286-3e73fe4d6bb6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:01.925828 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948f7578-1e2d-4db0-a286-3e73fe4d6bb6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:01.927998 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948f7578-1e2d-4db0-a286-3e73fe4d6bb6 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:01.942840 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 306/604] 10.222.0.45 () {56 vars in 1148 bytes} [Wed Aug 24 17:28:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:01.953915 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948f7578-1e2d-4db0-a286-3e73fe4d6bb6 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:01.988017 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e51f3e5-085d-4c63-8751-d206dca51147 None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:02.031013 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d64b6ef-01c2-4123-a25a-34512098dbb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:02.031836 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d64b6ef-01c2-4123-a25a-34512098dbb1 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:02.032116 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d64b6ef-01c2-4123-a25a-34512098dbb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:02.033636 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d64b6ef-01c2-4123-a25a-34512098dbb1 None admin] PATH_INFO: `/v3/domains` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:02.034300 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d64b6ef-01c2-4123-a25a-34512098dbb1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:02.034609 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d64b6ef-01c2-4123-a25a-34512098dbb1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:02.035986 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d64b6ef-01c2-4123-a25a-34512098dbb1 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:02.058243 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 299/605] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 24 17:28:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:02.065505 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d64b6ef-01c2-4123-a25a-34512098dbb1 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:02.112943 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:02.116054 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948f7578-1e2d-4db0-a286-3e73fe4d6bb6 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:02.160485 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7712fec0-7d99-4b10-99a0-296d7249b742 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:02.161257 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7712fec0-7d99-4b10-99a0-296d7249b742 None admin] REQUEST_METHOD: `PUT` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:02.161605 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7712fec0-7d99-4b10-99a0-296d7249b742 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:02.161965 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7712fec0-7d99-4b10-99a0-296d7249b742 None admin] PATH_INFO: `/v3/projects/2ad8a32bd9d2434097bd08b111914ca1/users/59be342541264942ad87f60ff4de6c8e/roles/ad6a81fb5d1941888577eb0ce9474882` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:02.172654 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:02.179871 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7712fec0-7d99-4b10-99a0-296d7249b742 None admin] RBAC: Authorizing `identity:create_grant(project_id=2ad8a32bd9d2434097bd08b111914ca1, user_id=59be342541264942ad87f60ff4de6c8e, role_id=ad6a81fb5d1941888577eb0ce9474882)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:02.182444 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7712fec0-7d99-4b10-99a0-296d7249b742 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:02.200626 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 307/606] 10.222.0.45 () {58 vars in 1439 bytes} [Wed Aug 24 17:28:01 2022] PUT /identity/v3/projects/2ad8a32bd9d2434097bd08b111914ca1/users/59be342541264942ad87f60ff4de6c8e/roles/ad6a81fb5d1941888577eb0ce9474882 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:28:02.201469 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7712fec0-7d99-4b10-99a0-296d7249b742 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:02.209547 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d64b6ef-01c2-4123-a25a-34512098dbb1 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:02.275763 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341a9d84-bcd9-4144-ab36-306e12d8323e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:02.276418 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341a9d84-bcd9-4144-ab36-306e12d8323e None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:02.276761 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341a9d84-bcd9-4144-ab36-306e12d8323e None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:02.277041 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341a9d84-bcd9-4144-ab36-306e12d8323e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:02.294225 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341a9d84-bcd9-4144-ab36-306e12d8323e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:02.296198 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341a9d84-bcd9-4144-ab36-306e12d8323e None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:02.322398 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application [None req-341a9d84-bcd9-4144-ab36-306e12d8323e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:28:02.322398 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:28:02.322398 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:28:02.322398 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:28:02.322398 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:28:02.322398 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:28:02.322398 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:28:02.322398 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:28:02.322398 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:28:02.322398 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:28:02.322398 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:28:02.322398 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:28:02.322398 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 17:28:02.322398 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 17:28:02.322398 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 17:28:02.323310 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 17:28:02.323310 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:28:02.323310 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:28:02.323310 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:28:02.323310 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:28:02.323310 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:28:02.323310 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:28:02.323310 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:28:02.323310 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application with Lock( Aug 24 17:28:02.323310 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:28:02.323310 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:28:02.323310 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:28:02.323310 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:28:02.323310 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:28:02.323310 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:28:02.323310 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:28:02.323310 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:28:02.324648 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 17:28:02.324648 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 17:28:02.324648 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 17:28:02.324648 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 17:28:02.324648 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 17:28:02.324648 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 17:28:02.324648 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:28:02.324648 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Aug 24 17:28:02.325795 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7712fec0-7d99-4b10-99a0-296d7249b742 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:02.325945 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 300/607] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Aug 24 17:28:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:28:02.329279 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341a9d84-bcd9-4144-ab36-306e12d8323e None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:02.363493 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03905c2-f497-4de5-b621-bb5120285e24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:02.364141 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03905c2-f497-4de5-b621-bb5120285e24 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:02.364416 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03905c2-f497-4de5-b621-bb5120285e24 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:02.364716 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03905c2-f497-4de5-b621-bb5120285e24 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:02.379216 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03905c2-f497-4de5-b621-bb5120285e24 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:28:02.384582 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application [None req-d03905c2-f497-4de5-b621-bb5120285e24 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 17:28:02.384582 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:28:02.384582 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:28:02.384582 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:28:02.384582 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:28:02.384582 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:28:02.384582 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:28:02.384582 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:28:02.384582 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:28:02.384582 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:28:02.384582 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:28:02.384582 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:28:02.384582 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:28:02.384582 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:28:02.384582 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:28:02.384582 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:28:02.387435 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:28:02.387435 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:28:02.387435 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:28:02.387435 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:28:02.387435 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:28:02.387435 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:28:02.387435 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:28:02.387435 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:28:02.387435 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:28:02.387435 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:28:02.387435 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:28:02.387435 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application with Lock( Aug 24 17:28:02.387435 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:28:02.387435 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:28:02.387435 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:28:02.387435 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:28:02.389525 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:28:02.389525 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:28:02.389525 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:28:02.389525 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:28:02.389525 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:28:02.389525 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:28:02.389525 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:28:02.389525 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:28:02.389525 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:28:02.389525 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:28:02.389525 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 17:28:02.389525 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Aug 24 17:28:02.389525 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 308/608] 10.222.0.45 () {56 vars in 1102 bytes} [Wed Aug 24 17:28:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:28:02.394182 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03905c2-f497-4de5-b621-bb5120285e24 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:02.407524 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:02.526761 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03905c2-f497-4de5-b621-bb5120285e24 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:02.529729 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341a9d84-bcd9-4144-ab36-306e12d8323e None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:02.571210 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d33e9c7-7cdf-4d85-8178-daba1b212243 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:02.571990 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d33e9c7-7cdf-4d85-8178-daba1b212243 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:02.572365 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d33e9c7-7cdf-4d85-8178-daba1b212243 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:02.573354 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d33e9c7-7cdf-4d85-8178-daba1b212243 None admin] PATH_INFO: `/v3/projects` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:02.574104 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d33e9c7-7cdf-4d85-8178-daba1b212243 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:02.574511 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d33e9c7-7cdf-4d85-8178-daba1b212243 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:02.574968 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9caad0c-d37b-4b27-818f-cbf275494a99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:02.575799 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9caad0c-d37b-4b27-818f-cbf275494a99 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:02.576236 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d33e9c7-7cdf-4d85-8178-daba1b212243 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:02.576363 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9caad0c-d37b-4b27-818f-cbf275494a99 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:02.576825 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9caad0c-d37b-4b27-818f-cbf275494a99 None admin] PATH_INFO: `/v3/roles` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:02.577624 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9caad0c-d37b-4b27-818f-cbf275494a99 None admin] RBAC: Adding query filter params (name=service) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:02.578075 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9caad0c-d37b-4b27-818f-cbf275494a99 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:02.579093 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:02.579803 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9caad0c-d37b-4b27-818f-cbf275494a99 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:02.590457 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 309/609] 10.222.0.45 () {56 vars in 1103 bytes} [Wed Aug 24 17:28:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:28:02.598012 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 301/610] 10.222.0.45 () {56 vars in 1148 bytes} [Wed Aug 24 17:28:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:28:02.599576 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9caad0c-d37b-4b27-818f-cbf275494a99 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:02.599693 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d33e9c7-7cdf-4d85-8178-daba1b212243 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:02.719431 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:02.771243 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d33e9c7-7cdf-4d85-8178-daba1b212243 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:02.774413 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9caad0c-d37b-4b27-818f-cbf275494a99 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:02.832940 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02b54f4-10ac-4328-94e5-299cc48ae184 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:02.833530 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02b54f4-10ac-4328-94e5-299cc48ae184 None admin] REQUEST_METHOD: `PUT` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:02.833811 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02b54f4-10ac-4328-94e5-299cc48ae184 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:02.834085 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02b54f4-10ac-4328-94e5-299cc48ae184 None admin] PATH_INFO: `/v3/projects/2ad8a32bd9d2434097bd08b111914ca1/users/43ad98a250a144099ccd18378aedd4b5/roles/ad6a81fb5d1941888577eb0ce9474882` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:02.838002 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93490b6c-5820-4296-909a-832ef4c49486 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:02.839156 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93490b6c-5820-4296-909a-832ef4c49486 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:02.839467 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93490b6c-5820-4296-909a-832ef4c49486 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:02.839741 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93490b6c-5820-4296-909a-832ef4c49486 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:02.858948 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93490b6c-5820-4296-909a-832ef4c49486 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:02.860406 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93490b6c-5820-4296-909a-832ef4c49486 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:02.862541 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02b54f4-10ac-4328-94e5-299cc48ae184 None admin] RBAC: Authorizing `identity:create_grant(project_id=2ad8a32bd9d2434097bd08b111914ca1, user_id=43ad98a250a144099ccd18378aedd4b5, role_id=ad6a81fb5d1941888577eb0ce9474882)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:02.863969 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02b54f4-10ac-4328-94e5-299cc48ae184 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:02.882990 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 310/611] 10.222.0.45 () {58 vars in 1439 bytes} [Wed Aug 24 17:28:02 2022] PUT /identity/v3/projects/2ad8a32bd9d2434097bd08b111914ca1/users/43ad98a250a144099ccd18378aedd4b5/roles/ad6a81fb5d1941888577eb0ce9474882 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:28:02.894538 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application [None req-93490b6c-5820-4296-909a-832ef4c49486 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:28:02.894538 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:28:02.894538 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:28:02.894538 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:28:02.894538 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:28:02.894538 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:28:02.894538 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:28:02.894538 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:28:02.894538 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:28:02.894538 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:28:02.894538 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:28:02.894538 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:28:02.894538 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:28:02.894538 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:28:02.894538 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:28:02.894538 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:28:02.895549 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:28:02.895549 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:28:02.895549 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:28:02.895549 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:28:02.895549 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:28:02.895549 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:28:02.895549 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:28:02.895549 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application with Lock( Aug 24 17:28:02.895549 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:28:02.895549 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:28:02.895549 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:28:02.895549 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:28:02.895549 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:28:02.895549 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:28:02.895549 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:28:02.895549 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:28:02.895549 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:28:02.897044 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:28:02.897044 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:28:02.897044 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Aug 24 17:28:02.897044 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 302/612] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 17:28:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:28:02.902332 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02b54f4-10ac-4328-94e5-299cc48ae184 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:03.068510 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02b54f4-10ac-4328-94e5-299cc48ae184 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:03.090786 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:03.112131 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94728d64-971f-4aa3-85c4-52d6a7aba134 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:03.112835 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94728d64-971f-4aa3-85c4-52d6a7aba134 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:03.113116 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94728d64-971f-4aa3-85c4-52d6a7aba134 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:03.113396 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94728d64-971f-4aa3-85c4-52d6a7aba134 None admin] PATH_INFO: `/v3/domains` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:03.114047 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94728d64-971f-4aa3-85c4-52d6a7aba134 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:03.114352 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94728d64-971f-4aa3-85c4-52d6a7aba134 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:03.115776 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94728d64-971f-4aa3-85c4-52d6a7aba134 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:03.128826 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 311/613] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 24 17:28:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:03.135951 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93490b6c-5820-4296-909a-832ef4c49486 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:03.189865 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:03.254765 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93490b6c-5820-4296-909a-832ef4c49486 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:03.297851 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b906d9e9-3035-40db-b7fd-41fa97383750 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:03.298487 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b906d9e9-3035-40db-b7fd-41fa97383750 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:03.298765 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b906d9e9-3035-40db-b7fd-41fa97383750 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:03.299039 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b906d9e9-3035-40db-b7fd-41fa97383750 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:03.306658 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b906d9e9-3035-40db-b7fd-41fa97383750 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:03.308688 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b906d9e9-3035-40db-b7fd-41fa97383750 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:03.320033 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application [None req-b906d9e9-3035-40db-b7fd-41fa97383750 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 17:28:03.320033 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:28:03.320033 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:28:03.320033 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:28:03.320033 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:28:03.320033 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:28:03.320033 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:28:03.320033 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:28:03.320033 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:28:03.320033 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:28:03.320033 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:28:03.320033 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:28:03.320033 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 17:28:03.320033 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 17:28:03.320033 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 17:28:03.320033 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 17:28:03.321021 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:28:03.321021 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:28:03.321021 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 17:28:03.321021 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:28:03.321021 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 17:28:03.321021 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:28:03.321021 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:28:03.321021 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:28:03.321021 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:28:03.321021 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:28:03.321021 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:28:03.321021 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application with Lock( Aug 24 17:28:03.321021 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:28:03.321021 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:28:03.321021 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:28:03.321021 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:28:03.321021 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:28:03.321021 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:28:03.322295 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:28:03.322295 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:28:03.322295 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 17:28:03.322295 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 17:28:03.322295 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 17:28:03.322295 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 17:28:03.322295 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 17:28:03.322295 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 17:28:03.322295 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 17:28:03.322295 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Aug 24 17:28:03.322295 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 303/614] 10.222.0.45 () {56 vars in 1096 bytes} [Wed Aug 24 17:28:03 2022] GET /identity/v3/users/swift => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:28:03.330013 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94728d64-971f-4aa3-85c4-52d6a7aba134 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:03.441542 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94728d64-971f-4aa3-85c4-52d6a7aba134 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:03.477694 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b72387-4128-45bd-b6c3-3f18318ddbfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:03.478306 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b72387-4128-45bd-b6c3-3f18318ddbfb None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:03.478588 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b72387-4128-45bd-b6c3-3f18318ddbfb None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:03.478918 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b72387-4128-45bd-b6c3-3f18318ddbfb None admin] PATH_INFO: `/v3/users` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:03.480414 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b72387-4128-45bd-b6c3-3f18318ddbfb None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:03.480749 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b72387-4128-45bd-b6c3-3f18318ddbfb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:03.482251 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b72387-4128-45bd-b6c3-3f18318ddbfb None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:03.499901 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 312/615] 10.222.0.45 () {56 vars in 1135 bytes} [Wed Aug 24 17:28:03 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:03.505048 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b906d9e9-3035-40db-b7fd-41fa97383750 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:03.580581 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec40ee23-db93-44b9-af11-70b4fdd2c37f None None] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:03.581091 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec40ee23-db93-44b9-af11-70b4fdd2c37f None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:03.581196 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec40ee23-db93-44b9-af11-70b4fdd2c37f None None] PATH_INFO: `/` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:03.582456 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 313/616] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:28:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:28:03.607470 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de2bf8e-f135-48e1-b720-9ed11b70ca04 None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:03.607754 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de2bf8e-f135-48e1-b720-9ed11b70ca04 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:03.608028 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de2bf8e-f135-48e1-b720-9ed11b70ca04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:03.626318 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-2de2bf8e-f135-48e1-b720-9ed11b70ca04 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:28:03.627509 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-2de2bf8e-f135-48e1-b720-9ed11b70ca04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:28:03.638206 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b906d9e9-3035-40db-b7fd-41fa97383750 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:03.645548 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:03.658950 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:03.676973 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f32502b-14a5-4c23-93ca-be3dbdf7d7de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:03.677603 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f32502b-14a5-4c23-93ca-be3dbdf7d7de None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:03.677879 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f32502b-14a5-4c23-93ca-be3dbdf7d7de None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:03.678151 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f32502b-14a5-4c23-93ca-be3dbdf7d7de None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:03.688856 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f32502b-14a5-4c23-93ca-be3dbdf7d7de None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:03.690293 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f32502b-14a5-4c23-93ca-be3dbdf7d7de None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:03.701589 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 314/617] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:28:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:28:03.702796 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbcb166-865a-48f5-a9cf-ef2ae8810a18 None None] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:03.703079 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbcb166-865a-48f5-a9cf-ef2ae8810a18 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:03.703350 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbcb166-865a-48f5-a9cf-ef2ae8810a18 None None] PATH_INFO: `/` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:03.704816 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 315/618] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:28:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:28:03.708830 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application [None req-5f32502b-14a5-4c23-93ca-be3dbdf7d7de None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:28:03.708830 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:28:03.708830 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:28:03.708830 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:28:03.708830 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:28:03.708830 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:28:03.708830 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:28:03.708830 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:28:03.708830 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:28:03.708830 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:28:03.708830 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:28:03.708830 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:28:03.708830 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:28:03.708830 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:28:03.708830 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:28:03.708830 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:28:03.709837 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:28:03.709837 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:28:03.709837 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:28:03.709837 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:28:03.709837 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:28:03.709837 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:28:03.709837 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:28:03.709837 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application with Lock( Aug 24 17:28:03.709837 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:28:03.709837 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:28:03.709837 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:28:03.709837 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:28:03.709837 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:28:03.709837 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:28:03.709837 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:28:03.709837 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:28:03.709837 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:28:03.710695 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:28:03.710695 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:28:03.710695 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Aug 24 17:28:03.711253 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 304/619] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 17:28:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 17:28:03.716225 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbcb166-865a-48f5-a9cf-ef2ae8810a18 None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:03.716659 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363cf87d-a604-4823-ac1d-c3fc9c235372 None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:03.716946 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363cf87d-a604-4823-ac1d-c3fc9c235372 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:03.721943 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363cf87d-a604-4823-ac1d-c3fc9c235372 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:03.748091 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-363cf87d-a604-4823-ac1d-c3fc9c235372 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:28:03.749307 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-363cf87d-a604-4823-ac1d-c3fc9c235372 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:28:03.872697 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 305/620] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:28:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 17:28:03.942196 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbcb166-865a-48f5-a9cf-ef2ae8810a18 None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:03.992376 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d028f5-0f68-4014-8a7b-cfc3ee995c70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:03.992927 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d028f5-0f68-4014-8a7b-cfc3ee995c70 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:03.993205 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d028f5-0f68-4014-8a7b-cfc3ee995c70 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:03.993472 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d028f5-0f68-4014-8a7b-cfc3ee995c70 None admin] PATH_INFO: `/v3/domains` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:03.994115 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d028f5-0f68-4014-8a7b-cfc3ee995c70 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:03.994448 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d028f5-0f68-4014-8a7b-cfc3ee995c70 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:03.996335 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d028f5-0f68-4014-8a7b-cfc3ee995c70 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:04.012653 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 316/621] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 24 17:28:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:04.014091 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363cf87d-a604-4823-ac1d-c3fc9c235372 None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:04.027441 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d028f5-0f68-4014-8a7b-cfc3ee995c70 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:04.086785 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:04.132915 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:04.138676 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363cf87d-a604-4823-ac1d-c3fc9c235372 None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:04.174184 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2cab9e-1249-4d96-b614-11cea116d1eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:04.174871 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2cab9e-1249-4d96-b614-11cea116d1eb None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:04.175146 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2cab9e-1249-4d96-b614-11cea116d1eb None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:04.175453 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2cab9e-1249-4d96-b614-11cea116d1eb None admin] PATH_INFO: `/v3/projects/service` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:04.182332 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2cab9e-1249-4d96-b614-11cea116d1eb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:04.183832 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2cab9e-1249-4d96-b614-11cea116d1eb None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:04.197574 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application [None req-ba2cab9e-1249-4d96-b614-11cea116d1eb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:28:04.197574 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:28:04.197574 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:28:04.197574 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:28:04.197574 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:28:04.197574 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:28:04.197574 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:28:04.197574 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:28:04.197574 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:28:04.197574 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:28:04.197574 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:28:04.197574 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:28:04.197574 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 17:28:04.197574 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 17:28:04.197574 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 17:28:04.198707 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 17:28:04.198707 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:28:04.198707 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:28:04.198707 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:28:04.198707 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:28:04.198707 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:28:04.198707 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:28:04.198707 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:28:04.198707 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application with Lock( Aug 24 17:28:04.198707 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:28:04.198707 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:28:04.198707 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:28:04.198707 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:28:04.198707 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:28:04.198707 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:28:04.198707 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:28:04.198707 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:28:04.199802 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 17:28:04.199802 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 17:28:04.199802 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 17:28:04.199802 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 17:28:04.199802 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 17:28:04.199802 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 17:28:04.199802 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:28:04.199802 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Aug 24 17:28:04.206344 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 306/622] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Aug 24 17:28:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:28:04.207325 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2cab9e-1249-4d96-b614-11cea116d1eb None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:04.237374 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d028f5-0f68-4014-8a7b-cfc3ee995c70 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:04.273333 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9f6e0d-cb99-4911-8f61-b851f9787950 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:04.274304 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9f6e0d-cb99-4911-8f61-b851f9787950 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:04.274725 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9f6e0d-cb99-4911-8f61-b851f9787950 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:04.275250 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9f6e0d-cb99-4911-8f61-b851f9787950 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:04.286770 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9f6e0d-cb99-4911-8f61-b851f9787950 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:28:04.292986 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application [None req-db9f6e0d-cb99-4911-8f61-b851f9787950 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 17:28:04.292986 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:28:04.292986 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:28:04.292986 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:28:04.292986 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:28:04.292986 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:28:04.292986 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:28:04.292986 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:28:04.292986 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:28:04.292986 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:28:04.292986 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:28:04.292986 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:28:04.292986 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:28:04.292986 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:28:04.292986 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:28:04.292986 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:28:04.293900 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:28:04.293900 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:28:04.293900 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:28:04.293900 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:28:04.293900 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:28:04.293900 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:28:04.293900 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:28:04.293900 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:28:04.293900 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:28:04.293900 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:28:04.293900 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:28:04.293900 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application with Lock( Aug 24 17:28:04.293900 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:28:04.293900 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:28:04.293900 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:28:04.293900 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:28:04.294731 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:28:04.294731 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:28:04.294731 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:28:04.294731 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:28:04.294731 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:28:04.294731 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:28:04.294731 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:28:04.294731 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:28:04.294731 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:28:04.294731 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:28:04.294731 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 17:28:04.294731 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Aug 24 17:28:04.301799 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 317/623] 10.222.0.45 () {56 vars in 1102 bytes} [Wed Aug 24 17:28:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:28:04.302960 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9f6e0d-cb99-4911-8f61-b851f9787950 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:04.310074 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2cab9e-1249-4d96-b614-11cea116d1eb None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:04.344321 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4a05c6-bc92-4e7c-9c95-3131624fea00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:04.344989 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4a05c6-bc92-4e7c-9c95-3131624fea00 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:04.345264 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4a05c6-bc92-4e7c-9c95-3131624fea00 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:04.345535 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4a05c6-bc92-4e7c-9c95-3131624fea00 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:04.356918 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4a05c6-bc92-4e7c-9c95-3131624fea00 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:28:04.361720 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application [None req-2c4a05c6-bc92-4e7c-9c95-3131624fea00 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 17:28:04.361720 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:28:04.361720 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:28:04.361720 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:28:04.361720 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:28:04.361720 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:28:04.361720 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:28:04.361720 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:28:04.361720 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:28:04.361720 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:28:04.361720 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:28:04.361720 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:28:04.361720 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:28:04.361720 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:28:04.361720 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:28:04.361720 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:28:04.362633 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:28:04.362633 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:28:04.362633 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:28:04.362633 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:28:04.362633 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:28:04.362633 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:28:04.362633 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:28:04.362633 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:28:04.362633 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:28:04.362633 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:28:04.362633 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:28:04.362633 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application with Lock( Aug 24 17:28:04.362633 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:28:04.362633 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:28:04.362633 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:28:04.362633 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:28:04.363445 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:28:04.363445 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:28:04.363445 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:28:04.363445 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:28:04.363445 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:28:04.363445 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:28:04.363445 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:28:04.363445 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:28:04.363445 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:28:04.363445 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:28:04.363445 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 17:28:04.363445 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Aug 24 17:28:04.363445 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 307/624] 10.222.0.45 () {56 vars in 1102 bytes} [Wed Aug 24 17:28:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:28:04.364119 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4a05c6-bc92-4e7c-9c95-3131624fea00 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:04.380924 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:04.411039 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:04.425469 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9f6e0d-cb99-4911-8f61-b851f9787950 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:04.461190 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53836e86-e7d9-4aa9-9c0e-9e1ac8c54f20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:04.462248 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53836e86-e7d9-4aa9-9c0e-9e1ac8c54f20 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:04.462742 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53836e86-e7d9-4aa9-9c0e-9e1ac8c54f20 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:04.463743 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53836e86-e7d9-4aa9-9c0e-9e1ac8c54f20 None admin] PATH_INFO: `/v3/projects` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:04.464672 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53836e86-e7d9-4aa9-9c0e-9e1ac8c54f20 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:04.465307 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53836e86-e7d9-4aa9-9c0e-9e1ac8c54f20 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:04.466934 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53836e86-e7d9-4aa9-9c0e-9e1ac8c54f20 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:04.477605 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4a05c6-bc92-4e7c-9c95-3131624fea00 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:04.481623 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 318/625] 10.222.0.45 () {56 vars in 1148 bytes} [Wed Aug 24 17:28:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:04.482664 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53836e86-e7d9-4aa9-9c0e-9e1ac8c54f20 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:04.521179 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbf29b6-2c37-47a4-95cb-c7f4244a1989 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:04.521799 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbf29b6-2c37-47a4-95cb-c7f4244a1989 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:04.522075 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbf29b6-2c37-47a4-95cb-c7f4244a1989 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:04.522345 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbf29b6-2c37-47a4-95cb-c7f4244a1989 None admin] PATH_INFO: `/v3/roles` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:04.523018 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbf29b6-2c37-47a4-95cb-c7f4244a1989 None admin] RBAC: Adding query filter params (name=service) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:04.523319 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbf29b6-2c37-47a4-95cb-c7f4244a1989 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:04.524786 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbf29b6-2c37-47a4-95cb-c7f4244a1989 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:04.539271 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 308/626] 10.222.0.45 () {56 vars in 1103 bytes} [Wed Aug 24 17:28:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:04.539271 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbf29b6-2c37-47a4-95cb-c7f4244a1989 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:04.596904 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53836e86-e7d9-4aa9-9c0e-9e1ac8c54f20 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:04.614498 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:04.636725 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597d5069-4b10-4332-8ff5-e4c0c455cce1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:04.637576 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597d5069-4b10-4332-8ff5-e4c0c455cce1 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:04.637999 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597d5069-4b10-4332-8ff5-e4c0c455cce1 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:04.638412 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597d5069-4b10-4332-8ff5-e4c0c455cce1 None admin] PATH_INFO: `/v3/roles` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:04.639346 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597d5069-4b10-4332-8ff5-e4c0c455cce1 None admin] RBAC: Adding query filter params (name=service) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:04.639787 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597d5069-4b10-4332-8ff5-e4c0c455cce1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:04.641429 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597d5069-4b10-4332-8ff5-e4c0c455cce1 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:04.649312 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbf29b6-2c37-47a4-95cb-c7f4244a1989 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:04.655892 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 319/627] 10.222.0.45 () {56 vars in 1103 bytes} [Wed Aug 24 17:28:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:04.656974 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597d5069-4b10-4332-8ff5-e4c0c455cce1 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:04.670032 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:04.686945 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12becca-ba37-4316-b5d4-48acb3ab851c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:04.687905 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12becca-ba37-4316-b5d4-48acb3ab851c None admin] REQUEST_METHOD: `PUT` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:04.688380 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12becca-ba37-4316-b5d4-48acb3ab851c None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:04.688882 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12becca-ba37-4316-b5d4-48acb3ab851c None admin] PATH_INFO: `/v3/projects/2ad8a32bd9d2434097bd08b111914ca1/users/90f0772b64c84faaa689e7272fa97898/roles/ad6a81fb5d1941888577eb0ce9474882` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:04.705076 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12becca-ba37-4316-b5d4-48acb3ab851c None admin] RBAC: Authorizing `identity:create_grant(project_id=2ad8a32bd9d2434097bd08b111914ca1, user_id=90f0772b64c84faaa689e7272fa97898, role_id=ad6a81fb5d1941888577eb0ce9474882)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:04.706907 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12becca-ba37-4316-b5d4-48acb3ab851c None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:04.728740 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 309/628] 10.222.0.45 () {58 vars in 1439 bytes} [Wed Aug 24 17:28:04 2022] PUT /identity/v3/projects/2ad8a32bd9d2434097bd08b111914ca1/users/90f0772b64c84faaa689e7272fa97898/roles/ad6a81fb5d1941888577eb0ce9474882 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:28:04.730346 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12becca-ba37-4316-b5d4-48acb3ab851c None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:04.769380 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597d5069-4b10-4332-8ff5-e4c0c455cce1 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:04.821997 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b66c10-7e2b-4daf-bf06-463f73260db3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:04.823173 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b66c10-7e2b-4daf-bf06-463f73260db3 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:04.823454 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b66c10-7e2b-4daf-bf06-463f73260db3 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:04.823724 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b66c10-7e2b-4daf-bf06-463f73260db3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:04.848581 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b66c10-7e2b-4daf-bf06-463f73260db3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:04.848581 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b66c10-7e2b-4daf-bf06-463f73260db3 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:04.861860 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application [None req-62b66c10-7e2b-4daf-bf06-463f73260db3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:28:04.861860 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:28:04.861860 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:28:04.861860 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:28:04.861860 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:28:04.861860 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:28:04.861860 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:28:04.861860 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:28:04.861860 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:28:04.861860 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:28:04.861860 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:28:04.861860 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:28:04.861860 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:28:04.861860 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:28:04.861860 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:28:04.861860 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:28:04.862854 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:28:04.862854 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:28:04.862854 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:28:04.862854 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:28:04.862854 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:28:04.862854 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:28:04.862854 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:28:04.862854 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application with Lock( Aug 24 17:28:04.862854 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:28:04.862854 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:28:04.862854 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:28:04.862854 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:28:04.862854 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:28:04.862854 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:28:04.862854 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:28:04.862854 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:28:04.862854 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:28:04.864365 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:28:04.864365 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:28:04.864365 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Aug 24 17:28:04.864365 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 320/629] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 17:28:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:28:04.869655 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d60463-7b2a-412f-a35d-87363917609a None None] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:04.869948 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d60463-7b2a-412f-a35d-87363917609a None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:04.870214 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d60463-7b2a-412f-a35d-87363917609a None None] PATH_INFO: `/` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:04.871856 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 321/630] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:28:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:28:04.872639 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d60463-7b2a-412f-a35d-87363917609a None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:04.894733 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12becca-ba37-4316-b5d4-48acb3ab851c None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:04.919202 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:04.940321 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5986b98b-52f1-4e7f-b20c-de1a557f6e66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:04.941160 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5986b98b-52f1-4e7f-b20c-de1a557f6e66 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:04.941572 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5986b98b-52f1-4e7f-b20c-de1a557f6e66 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:04.942035 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5986b98b-52f1-4e7f-b20c-de1a557f6e66 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:04.949483 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:04.955339 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5986b98b-52f1-4e7f-b20c-de1a557f6e66 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:04.956845 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5986b98b-52f1-4e7f-b20c-de1a557f6e66 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:04.975080 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application [None req-5986b98b-52f1-4e7f-b20c-de1a557f6e66 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:28:04.975080 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:28:04.975080 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:28:04.975080 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:28:04.975080 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:28:04.975080 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:28:04.975080 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:28:04.975080 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:28:04.975080 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:28:04.975080 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:28:04.975080 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:28:04.975080 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:28:04.975080 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:28:04.975080 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:28:04.975080 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:28:04.975080 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:28:04.976049 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:28:04.976049 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:28:04.976049 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:28:04.976049 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:28:04.976049 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:28:04.976049 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:28:04.976049 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:28:04.976049 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application with Lock( Aug 24 17:28:04.976049 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:28:04.976049 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:28:04.976049 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:28:04.976049 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:28:04.976049 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:28:04.976049 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:28:04.976049 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:28:04.976049 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:28:04.976049 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:28:04.977823 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:28:04.977823 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:28:04.977823 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Aug 24 17:28:04.977823 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 310/631] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 17:28:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:28:04.977823 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ff6786-b658-4647-b907-f04443943b9a None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:04.977823 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ff6786-b658-4647-b907-f04443943b9a None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:04.978154 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ff6786-b658-4647-b907-f04443943b9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:04.997340 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-44ff6786-b658-4647-b907-f04443943b9a None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:28:04.998283 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-44ff6786-b658-4647-b907-f04443943b9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:28:05.000771 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d60463-7b2a-412f-a35d-87363917609a None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:05.038835 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53114139-a38a-4d20-8890-84e7d79f318d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:05.039451 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53114139-a38a-4d20-8890-84e7d79f318d None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:05.039916 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53114139-a38a-4d20-8890-84e7d79f318d None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:05.040199 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53114139-a38a-4d20-8890-84e7d79f318d None admin] PATH_INFO: `/v3/domains` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:05.040891 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53114139-a38a-4d20-8890-84e7d79f318d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:05.041305 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53114139-a38a-4d20-8890-84e7d79f318d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:05.042704 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53114139-a38a-4d20-8890-84e7d79f318d None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:05.054936 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 322/632] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 24 17:28:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:05.055599 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53114139-a38a-4d20-8890-84e7d79f318d None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:05.077264 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 311/633] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:28:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:28:05.079141 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f47af61-a32b-4671-9ef7-94d1fa1b67d0 None None] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:05.079427 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f47af61-a32b-4671-9ef7-94d1fa1b67d0 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:05.079700 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f47af61-a32b-4671-9ef7-94d1fa1b67d0 None None] PATH_INFO: `/` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:05.084709 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 312/634] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:28:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:28:05.084709 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f47af61-a32b-4671-9ef7-94d1fa1b67d0 None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:05.180539 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53114139-a38a-4d20-8890-84e7d79f318d None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:05.212719 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:05.213019 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f47af61-a32b-4671-9ef7-94d1fa1b67d0 None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:05.216245 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:05.227627 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcdeeae-b7f7-46b9-b743-817494d7648c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:05.228236 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcdeeae-b7f7-46b9-b743-817494d7648c None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:05.228530 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcdeeae-b7f7-46b9-b743-817494d7648c None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:05.230152 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcdeeae-b7f7-46b9-b743-817494d7648c None admin] PATH_INFO: `/v3/domains` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:05.230804 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcdeeae-b7f7-46b9-b743-817494d7648c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:05.231121 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcdeeae-b7f7-46b9-b743-817494d7648c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:05.233453 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcdeeae-b7f7-46b9-b743-817494d7648c None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:05.252539 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 323/635] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 24 17:28:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:05.252539 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cf3339-224a-42c5-911d-9fe17be3de1f None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:05.252539 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cf3339-224a-42c5-911d-9fe17be3de1f None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:05.252539 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cf3339-224a-42c5-911d-9fe17be3de1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:05.263861 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b26fb53-c6c9-4eda-a0de-b0e8e9f2fafc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:05.264727 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b26fb53-c6c9-4eda-a0de-b0e8e9f2fafc None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:05.265739 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b26fb53-c6c9-4eda-a0de-b0e8e9f2fafc None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:05.266156 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b26fb53-c6c9-4eda-a0de-b0e8e9f2fafc None admin] PATH_INFO: `/v3/users/glance` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:05.270993 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-f6cf3339-224a-42c5-911d-9fe17be3de1f None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:28:05.272230 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-f6cf3339-224a-42c5-911d-9fe17be3de1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:28:05.274516 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b26fb53-c6c9-4eda-a0de-b0e8e9f2fafc None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:05.277633 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b26fb53-c6c9-4eda-a0de-b0e8e9f2fafc None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:05.295665 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application [None req-2b26fb53-c6c9-4eda-a0de-b0e8e9f2fafc None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 24 17:28:05.295665 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:28:05.295665 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:28:05.295665 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:28:05.295665 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:28:05.295665 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:28:05.295665 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:28:05.295665 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:28:05.295665 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:28:05.295665 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:28:05.295665 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:28:05.295665 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:28:05.295665 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 17:28:05.295665 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 17:28:05.295665 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 17:28:05.295665 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 17:28:05.296647 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:28:05.296647 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:28:05.296647 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 17:28:05.296647 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:28:05.296647 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 17:28:05.296647 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:28:05.296647 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:28:05.296647 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:28:05.296647 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:28:05.296647 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:28:05.296647 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:28:05.296647 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application with Lock( Aug 24 17:28:05.296647 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:28:05.296647 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:28:05.296647 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:28:05.296647 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:28:05.296647 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:28:05.296647 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:28:05.297570 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:28:05.297570 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:28:05.297570 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 17:28:05.297570 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 17:28:05.297570 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 17:28:05.297570 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 17:28:05.297570 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 17:28:05.297570 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 17:28:05.297570 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 24 17:28:05.297570 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Aug 24 17:28:05.300863 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 313/636] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Aug 24 17:28:05 2022] GET /identity/v3/users/glance => generated 84 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:28:05.300863 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b26fb53-c6c9-4eda-a0de-b0e8e9f2fafc None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:05.355584 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 324/637] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:28:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:28:05.356492 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cf3339-224a-42c5-911d-9fe17be3de1f None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:05.409403 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b26fb53-c6c9-4eda-a0de-b0e8e9f2fafc None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:05.456939 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff9dfb6-176b-47a3-9cc9-e42a497b2282 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:05.457662 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff9dfb6-176b-47a3-9cc9-e42a497b2282 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:05.458003 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff9dfb6-176b-47a3-9cc9-e42a497b2282 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:05.458343 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff9dfb6-176b-47a3-9cc9-e42a497b2282 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:05.469535 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff9dfb6-176b-47a3-9cc9-e42a497b2282 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:05.471701 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff9dfb6-176b-47a3-9cc9-e42a497b2282 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:05.473003 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cf3339-224a-42c5-911d-9fe17be3de1f None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:05.485791 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application [None req-cff9dfb6-176b-47a3-9cc9-e42a497b2282 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 17:28:05.485791 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:28:05.485791 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:28:05.485791 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:28:05.485791 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:28:05.485791 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:28:05.485791 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:28:05.485791 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:28:05.485791 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:28:05.485791 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:28:05.485791 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:28:05.485791 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:28:05.485791 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 17:28:05.485791 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 17:28:05.485791 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 17:28:05.485791 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 17:28:05.486757 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:28:05.486757 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:28:05.486757 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 17:28:05.486757 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:28:05.486757 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 17:28:05.486757 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:28:05.486757 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:28:05.486757 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:28:05.486757 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:28:05.486757 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:28:05.486757 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:28:05.486757 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application with Lock( Aug 24 17:28:05.486757 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:28:05.486757 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:28:05.486757 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:28:05.486757 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:28:05.486757 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:28:05.486757 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:28:05.487679 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:28:05.487679 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:28:05.487679 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 17:28:05.487679 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 17:28:05.487679 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 17:28:05.487679 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 17:28:05.487679 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 17:28:05.487679 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 17:28:05.487679 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 17:28:05.487679 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Aug 24 17:28:05.491605 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 314/638] 10.222.0.45 () {56 vars in 1093 bytes} [Wed Aug 24 17:28:05 2022] GET /identity/v3/users/nova => generated 82 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:28:05.492516 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff9dfb6-176b-47a3-9cc9-e42a497b2282 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:05.493841 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:05.520181 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b099452-264a-42e9-b06a-459a175522cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:05.521157 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b099452-264a-42e9-b06a-459a175522cd None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:05.521636 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b099452-264a-42e9-b06a-459a175522cd None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:05.522133 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b099452-264a-42e9-b06a-459a175522cd None admin] PATH_INFO: `/v3/users` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:05.523163 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b099452-264a-42e9-b06a-459a175522cd None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:05.523699 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b099452-264a-42e9-b06a-459a175522cd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:05.525066 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:05.527488 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b099452-264a-42e9-b06a-459a175522cd None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:05.547806 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 325/639] 10.222.0.45 () {56 vars in 1137 bytes} [Wed Aug 24 17:28:05 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:05.549062 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b099452-264a-42e9-b06a-459a175522cd None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:05.619507 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff9dfb6-176b-47a3-9cc9-e42a497b2282 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:05.656760 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b31264-56bb-4a0d-bdeb-0297a0e62999 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:05.657473 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b31264-56bb-4a0d-bdeb-0297a0e62999 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:05.657823 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b31264-56bb-4a0d-bdeb-0297a0e62999 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:05.658098 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b31264-56bb-4a0d-bdeb-0297a0e62999 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:05.663319 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b099452-264a-42e9-b06a-459a175522cd None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:05.668239 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b31264-56bb-4a0d-bdeb-0297a0e62999 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:28:05.673292 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application [None req-87b31264-56bb-4a0d-bdeb-0297a0e62999 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 17:28:05.673292 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:28:05.673292 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:28:05.673292 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:28:05.673292 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:28:05.673292 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:28:05.673292 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:28:05.673292 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:28:05.673292 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:28:05.673292 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:28:05.673292 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:28:05.673292 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:28:05.673292 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:28:05.673292 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:28:05.673292 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:28:05.673292 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:28:05.674212 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:28:05.674212 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:28:05.674212 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:28:05.674212 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:28:05.674212 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:28:05.674212 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:28:05.674212 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:28:05.674212 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:28:05.674212 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:28:05.674212 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:28:05.674212 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:28:05.674212 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application with Lock( Aug 24 17:28:05.674212 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:28:05.674212 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:28:05.674212 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:28:05.674212 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:28:05.675012 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:28:05.675012 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:28:05.675012 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:28:05.675012 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:28:05.675012 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:28:05.675012 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:28:05.675012 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:28:05.675012 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:28:05.675012 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:28:05.675012 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:28:05.675012 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 17:28:05.675012 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Aug 24 17:28:05.675012 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 315/640] 10.222.0.45 () {56 vars in 1102 bytes} [Wed Aug 24 17:28:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:28:05.675687 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b31264-56bb-4a0d-bdeb-0297a0e62999 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:05.704587 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085914ee-b6f2-4186-b4c4-c10728fa7cd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:05.705196 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085914ee-b6f2-4186-b4c4-c10728fa7cd9 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:05.705475 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085914ee-b6f2-4186-b4c4-c10728fa7cd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:05.705742 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085914ee-b6f2-4186-b4c4-c10728fa7cd9 None admin] PATH_INFO: `/v3/users` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:05.706454 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085914ee-b6f2-4186-b4c4-c10728fa7cd9 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:05.706754 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085914ee-b6f2-4186-b4c4-c10728fa7cd9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:05.708173 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085914ee-b6f2-4186-b4c4-c10728fa7cd9 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:05.729536 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 326/641] 10.222.0.45 () {56 vars in 1133 bytes} [Wed Aug 24 17:28:05 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:05.729536 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085914ee-b6f2-4186-b4c4-c10728fa7cd9 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:05.753537 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:05.766974 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:05.789522 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b31264-56bb-4a0d-bdeb-0297a0e62999 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:05.829040 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a31d79a-46cb-43a8-be22-bc00579862fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:05.829858 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a31d79a-46cb-43a8-be22-bc00579862fc None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:05.829978 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a31d79a-46cb-43a8-be22-bc00579862fc None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:05.830179 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a31d79a-46cb-43a8-be22-bc00579862fc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:05.833026 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085914ee-b6f2-4186-b4c4-c10728fa7cd9 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:05.867843 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41673311-0b82-462e-bbaa-75c680f754d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:05.868481 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41673311-0b82-462e-bbaa-75c680f754d6 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:05.868786 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41673311-0b82-462e-bbaa-75c680f754d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:05.869061 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41673311-0b82-462e-bbaa-75c680f754d6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:05.880267 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41673311-0b82-462e-bbaa-75c680f754d6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:28:05.885742 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application [None req-41673311-0b82-462e-bbaa-75c680f754d6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 17:28:05.885742 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:28:05.885742 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:28:05.885742 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:28:05.885742 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:28:05.885742 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:28:05.885742 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:28:05.885742 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:28:05.885742 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:28:05.885742 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:28:05.885742 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:28:05.885742 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:28:05.885742 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:28:05.885742 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:28:05.885742 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:28:05.885742 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:28:05.886948 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:28:05.886948 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:28:05.886948 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:28:05.886948 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:28:05.886948 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:28:05.886948 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:28:05.886948 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:28:05.886948 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:28:05.886948 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:28:05.886948 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:28:05.886948 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:28:05.886948 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application with Lock( Aug 24 17:28:05.886948 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:28:05.886948 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:28:05.886948 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:28:05.886948 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:28:05.887976 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:28:05.887976 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:28:05.887976 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:28:05.887976 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:28:05.887976 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:28:05.887976 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:28:05.887976 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:28:05.887976 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:28:05.887976 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:28:05.887976 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:28:05.887976 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 17:28:05.887976 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Aug 24 17:28:05.889924 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 327/642] 10.222.0.45 () {56 vars in 1102 bytes} [Wed Aug 24 17:28:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:28:05.890826 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41673311-0b82-462e-bbaa-75c680f754d6 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:05.907242 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a31d79a-46cb-43a8-be22-bc00579862fc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:05.909025 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a31d79a-46cb-43a8-be22-bc00579862fc None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:05.981899 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:05.995308 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41673311-0b82-462e-bbaa-75c680f754d6 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:06.012060 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:06.031393 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76728f0-f3b1-445d-aeca-3c38ea40021d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:06.031999 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76728f0-f3b1-445d-aeca-3c38ea40021d None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:06.032281 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76728f0-f3b1-445d-aeca-3c38ea40021d None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:06.032578 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76728f0-f3b1-445d-aeca-3c38ea40021d None admin] PATH_INFO: `/v3/roles` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:06.033249 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76728f0-f3b1-445d-aeca-3c38ea40021d None admin] RBAC: Adding query filter params (name=service) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:06.033556 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76728f0-f3b1-445d-aeca-3c38ea40021d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:06.034946 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76728f0-f3b1-445d-aeca-3c38ea40021d None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:06.040971 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 316/643] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Aug 24 17:28:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 17:28:06.040971 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a31d79a-46cb-43a8-be22-bc00579862fc None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:06.044613 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 328/644] 10.222.0.45 () {56 vars in 1103 bytes} [Wed Aug 24 17:28:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:06.045567 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76728f0-f3b1-445d-aeca-3c38ea40021d None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:06.141439 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a31d79a-46cb-43a8-be22-bc00579862fc None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:06.158702 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76728f0-f3b1-445d-aeca-3c38ea40021d None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:06.179171 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b23b568-2e82-4c24-b84b-110471929686 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:06.180172 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b23b568-2e82-4c24-b84b-110471929686 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:06.180545 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b23b568-2e82-4c24-b84b-110471929686 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:06.180939 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b23b568-2e82-4c24-b84b-110471929686 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:06.192777 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0223dc02-aee8-422c-b0d3-67c1bdf44181 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:06.193386 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0223dc02-aee8-422c-b0d3-67c1bdf44181 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:06.193666 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0223dc02-aee8-422c-b0d3-67c1bdf44181 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:06.193943 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0223dc02-aee8-422c-b0d3-67c1bdf44181 None admin] PATH_INFO: `/v3/roles` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:06.194608 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0223dc02-aee8-422c-b0d3-67c1bdf44181 None admin] RBAC: Adding query filter params (name=service) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:06.194911 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0223dc02-aee8-422c-b0d3-67c1bdf44181 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:06.196268 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0223dc02-aee8-422c-b0d3-67c1bdf44181 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:06.208468 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 329/645] 10.222.0.45 () {56 vars in 1103 bytes} [Wed Aug 24 17:28:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:06.208468 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0223dc02-aee8-422c-b0d3-67c1bdf44181 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:06.223085 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:06.241061 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:06.260735 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b23b568-2e82-4c24-b84b-110471929686 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:06.262295 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b23b568-2e82-4c24-b84b-110471929686 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:06.314552 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0223dc02-aee8-422c-b0d3-67c1bdf44181 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:06.346794 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0874607-aea8-4774-8b53-11a61506c71f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:06.347423 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0874607-aea8-4774-8b53-11a61506c71f None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:06.347704 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0874607-aea8-4774-8b53-11a61506c71f None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:06.347976 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0874607-aea8-4774-8b53-11a61506c71f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:06.359858 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0874607-aea8-4774-8b53-11a61506c71f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:06.361338 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0874607-aea8-4774-8b53-11a61506c71f None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:06.371836 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 317/646] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Aug 24 17:28:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 17:28:06.371836 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b23b568-2e82-4c24-b84b-110471929686 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:06.380370 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application [None req-b0874607-aea8-4774-8b53-11a61506c71f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:28:06.380370 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:28:06.380370 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:28:06.380370 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:28:06.380370 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:28:06.380370 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:28:06.380370 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:28:06.380370 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:28:06.380370 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:28:06.380370 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:28:06.380370 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:28:06.380370 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:28:06.380370 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:28:06.380370 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:28:06.380370 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:28:06.380370 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:28:06.381338 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:28:06.381338 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:28:06.381338 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:28:06.381338 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:28:06.381338 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:28:06.381338 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:28:06.381338 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:28:06.381338 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application with Lock( Aug 24 17:28:06.381338 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:28:06.381338 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:28:06.381338 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:28:06.381338 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:28:06.381338 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:28:06.381338 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:28:06.381338 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:28:06.381338 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:28:06.381338 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:28:06.387169 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:28:06.387169 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:28:06.387169 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Aug 24 17:28:06.387169 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 330/647] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 17:28:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:28:06.387169 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0874607-aea8-4774-8b53-11a61506c71f None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:06.444144 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:06.463762 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:06.484088 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b23b568-2e82-4c24-b84b-110471929686 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:06.485082 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0874607-aea8-4774-8b53-11a61506c71f None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:06.520793 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60523fd3-3013-4be7-9982-a128c34776c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:06.521691 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60523fd3-3013-4be7-9982-a128c34776c4 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:06.521965 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60523fd3-3013-4be7-9982-a128c34776c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:06.522233 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60523fd3-3013-4be7-9982-a128c34776c4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:06.525237 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c017e02-6526-47d8-bf29-765cd3d4b7a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:06.526384 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c017e02-6526-47d8-bf29-765cd3d4b7a5 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:06.526667 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c017e02-6526-47d8-bf29-765cd3d4b7a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:06.526941 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c017e02-6526-47d8-bf29-765cd3d4b7a5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:06.533040 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60523fd3-3013-4be7-9982-a128c34776c4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:06.534526 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60523fd3-3013-4be7-9982-a128c34776c4 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:06.539920 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c017e02-6526-47d8-bf29-765cd3d4b7a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:06.542498 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c017e02-6526-47d8-bf29-765cd3d4b7a5 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:06.549606 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application [None req-60523fd3-3013-4be7-9982-a128c34776c4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:28:06.549606 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:28:06.549606 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:28:06.549606 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:28:06.549606 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:28:06.549606 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:28:06.549606 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:28:06.549606 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:28:06.549606 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:28:06.549606 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:28:06.549606 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:28:06.549606 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:28:06.549606 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:28:06.549606 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:28:06.549606 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:28:06.549606 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:28:06.550538 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:28:06.550538 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:28:06.550538 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:28:06.550538 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:28:06.550538 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:28:06.550538 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:28:06.550538 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:28:06.550538 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application with Lock( Aug 24 17:28:06.550538 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:28:06.550538 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:28:06.550538 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:28:06.550538 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:28:06.550538 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:28:06.550538 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:28:06.550538 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:28:06.550538 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:28:06.550538 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:28:06.551419 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:28:06.551419 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:28:06.551419 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Aug 24 17:28:06.555452 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 318/648] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 17:28:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:28:06.556270 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60523fd3-3013-4be7-9982-a128c34776c4 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:06.556667 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application [None req-5c017e02-6526-47d8-bf29-765cd3d4b7a5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:28:06.556667 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:28:06.556667 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:28:06.556667 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:28:06.556667 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:28:06.556667 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:28:06.556667 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:28:06.556667 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:28:06.556667 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:28:06.556667 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:28:06.556667 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:28:06.556667 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:28:06.556667 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:28:06.556667 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:28:06.556667 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:28:06.556667 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:28:06.557792 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:28:06.557792 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:28:06.557792 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:28:06.557792 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:28:06.557792 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:28:06.557792 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:28:06.557792 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:28:06.557792 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application with Lock( Aug 24 17:28:06.557792 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:28:06.557792 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:28:06.557792 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:28:06.557792 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:28:06.557792 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:28:06.557792 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:28:06.557792 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:28:06.557792 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:28:06.557792 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:28:06.558664 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:28:06.558664 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:28:06.558664 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Aug 24 17:28:06.558664 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 331/649] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 17:28:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:28:06.558881 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c017e02-6526-47d8-bf29-765cd3d4b7a5 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:06.662585 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60523fd3-3013-4be7-9982-a128c34776c4 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:06.664192 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c017e02-6526-47d8-bf29-765cd3d4b7a5 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:06.681260 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:06.697864 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f649313-83fd-4ea2-8dca-1c7ef5dea6d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:06.698425 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f649313-83fd-4ea2-8dca-1c7ef5dea6d7 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:06.698703 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f649313-83fd-4ea2-8dca-1c7ef5dea6d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:06.698972 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f649313-83fd-4ea2-8dca-1c7ef5dea6d7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:06.699797 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe31cfe9-b09d-4831-81a7-014db558045e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:06.700406 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe31cfe9-b09d-4831-81a7-014db558045e None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:06.700857 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:06.701489 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe31cfe9-b09d-4831-81a7-014db558045e None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:06.701866 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe31cfe9-b09d-4831-81a7-014db558045e None admin] PATH_INFO: `/v3/domains` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:06.702420 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe31cfe9-b09d-4831-81a7-014db558045e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:06.702724 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe31cfe9-b09d-4831-81a7-014db558045e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:06.704144 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe31cfe9-b09d-4831-81a7-014db558045e None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:06.710641 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f649313-83fd-4ea2-8dca-1c7ef5dea6d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:06.712057 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f649313-83fd-4ea2-8dca-1c7ef5dea6d7 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:06.720858 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 332/650] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 24 17:28:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:06.721573 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe31cfe9-b09d-4831-81a7-014db558045e None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:06.727164 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application [None req-8f649313-83fd-4ea2-8dca-1c7ef5dea6d7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:28:06.727164 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:28:06.727164 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:28:06.727164 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:28:06.727164 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:28:06.727164 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:28:06.727164 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:28:06.727164 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:28:06.727164 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:28:06.727164 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:28:06.727164 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:28:06.727164 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:28:06.727164 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:28:06.727164 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:28:06.727164 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:28:06.727164 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:28:06.728082 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:28:06.728082 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:28:06.728082 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:28:06.728082 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:28:06.728082 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:28:06.728082 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:28:06.728082 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:28:06.728082 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application with Lock( Aug 24 17:28:06.728082 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:28:06.728082 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:28:06.728082 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:28:06.728082 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:28:06.728082 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:28:06.728082 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:28:06.728082 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:28:06.728082 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:28:06.728082 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:28:06.729553 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:28:06.729553 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:28:06.729553 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Aug 24 17:28:06.735761 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 319/651] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 17:28:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:28:06.735761 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f649313-83fd-4ea2-8dca-1c7ef5dea6d7 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:06.822079 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe31cfe9-b09d-4831-81a7-014db558045e None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:06.833600 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f649313-83fd-4ea2-8dca-1c7ef5dea6d7 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:06.856635 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf5b9d3-3264-4c5b-b90d-173650e68f61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:06.857237 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf5b9d3-3264-4c5b-b90d-173650e68f61 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:06.857512 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf5b9d3-3264-4c5b-b90d-173650e68f61 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:06.857782 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf5b9d3-3264-4c5b-b90d-173650e68f61 None admin] PATH_INFO: `/v3/domains` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:06.858413 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf5b9d3-3264-4c5b-b90d-173650e68f61 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:06.858712 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf5b9d3-3264-4c5b-b90d-173650e68f61 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:06.860072 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf5b9d3-3264-4c5b-b90d-173650e68f61 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:06.867030 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a603bb-a74c-488f-b12f-dd2299ae1002 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:06.867633 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a603bb-a74c-488f-b12f-dd2299ae1002 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:06.867919 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a603bb-a74c-488f-b12f-dd2299ae1002 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:06.868201 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a603bb-a74c-488f-b12f-dd2299ae1002 None admin] PATH_INFO: `/v3/domains` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:06.868876 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a603bb-a74c-488f-b12f-dd2299ae1002 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:06.869180 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a603bb-a74c-488f-b12f-dd2299ae1002 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:06.870572 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a603bb-a74c-488f-b12f-dd2299ae1002 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:06.871376 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 333/652] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 24 17:28:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:06.873218 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf5b9d3-3264-4c5b-b90d-173650e68f61 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:06.882056 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 320/653] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 24 17:28:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:06.882862 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a603bb-a74c-488f-b12f-dd2299ae1002 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:06.908973 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:06.929435 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:06.974240 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf5b9d3-3264-4c5b-b90d-173650e68f61 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:06.980824 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a603bb-a74c-488f-b12f-dd2299ae1002 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:07.007159 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d24d5f-112c-424b-8940-8fe97839fd05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:07.007783 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d24d5f-112c-424b-8940-8fe97839fd05 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:07.008066 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d24d5f-112c-424b-8940-8fe97839fd05 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:07.008340 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d24d5f-112c-424b-8940-8fe97839fd05 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:07.013395 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbfa558-534e-4fa6-b6fb-cea98afe5475 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:07.014013 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbfa558-534e-4fa6-b6fb-cea98afe5475 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:07.014286 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbfa558-534e-4fa6-b6fb-cea98afe5475 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:07.014561 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbfa558-534e-4fa6-b6fb-cea98afe5475 None admin] PATH_INFO: `/v3/domains` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:07.015190 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbfa558-534e-4fa6-b6fb-cea98afe5475 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:07.015398 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d24d5f-112c-424b-8940-8fe97839fd05 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:07.015507 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbfa558-534e-4fa6-b6fb-cea98afe5475 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:07.016847 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbfa558-534e-4fa6-b6fb-cea98afe5475 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:07.016949 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d24d5f-112c-424b-8940-8fe97839fd05 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:07.028390 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application [None req-c3d24d5f-112c-424b-8940-8fe97839fd05 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:28:07.028390 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:28:07.028390 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:28:07.028390 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:28:07.028390 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:28:07.028390 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:28:07.028390 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:28:07.028390 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:28:07.028390 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:28:07.028390 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:28:07.028390 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:28:07.028390 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:28:07.028390 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 17:28:07.028390 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 17:28:07.028390 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 17:28:07.029298 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 17:28:07.029298 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:28:07.029298 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:28:07.029298 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:28:07.029298 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:28:07.029298 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:28:07.029298 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:28:07.029298 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:28:07.029298 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application with Lock( Aug 24 17:28:07.029298 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:28:07.029298 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:28:07.029298 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:28:07.029298 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:28:07.029298 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:28:07.029298 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:28:07.029298 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:28:07.029298 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:28:07.030160 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 17:28:07.030160 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 17:28:07.030160 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 17:28:07.030160 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 17:28:07.030160 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 17:28:07.030160 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 17:28:07.030160 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:28:07.030160 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Aug 24 17:28:07.030679 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 321/654] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 24 17:28:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:07.030679 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1f993c-f19d-4381-bf90-580510374c17 None None] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:07.030679 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1f993c-f19d-4381-bf90-580510374c17 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:07.030679 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1f993c-f19d-4381-bf90-580510374c17 None None] PATH_INFO: `/` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:07.031619 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 322/655] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:28:07 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:28:07.032218 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1f993c-f19d-4381-bf90-580510374c17 None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:07.032800 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 334/656] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Aug 24 17:28:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:28:07.033468 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d24d5f-112c-424b-8940-8fe97839fd05 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:07.125032 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:07.131647 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d24d5f-112c-424b-8940-8fe97839fd05 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:07.139309 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1f993c-f19d-4381-bf90-580510374c17 None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:07.146051 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:07.165458 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5a449a-c6f9-4b24-85b3-0fbb2bb3f4b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:07.166076 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5a449a-c6f9-4b24-85b3-0fbb2bb3f4b8 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:07.166353 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5a449a-c6f9-4b24-85b3-0fbb2bb3f4b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:07.166622 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5a449a-c6f9-4b24-85b3-0fbb2bb3f4b8 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:07.173247 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120aa9b9-a762-4d2d-954c-56da16abe0b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:07.173816 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5a449a-c6f9-4b24-85b3-0fbb2bb3f4b8 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:07.173920 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120aa9b9-a762-4d2d-954c-56da16abe0b4 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:07.174133 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120aa9b9-a762-4d2d-954c-56da16abe0b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:07.174408 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120aa9b9-a762-4d2d-954c-56da16abe0b4 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:07.175275 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5a449a-c6f9-4b24-85b3-0fbb2bb3f4b8 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:07.181250 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120aa9b9-a762-4d2d-954c-56da16abe0b4 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:07.183241 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120aa9b9-a762-4d2d-954c-56da16abe0b4 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:07.187041 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application [None req-3e5a449a-c6f9-4b24-85b3-0fbb2bb3f4b8 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 17:28:07.187041 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:28:07.187041 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:28:07.187041 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:28:07.187041 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:28:07.187041 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:28:07.187041 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:28:07.187041 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:28:07.187041 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:28:07.187041 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:28:07.187041 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:28:07.187041 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:28:07.187041 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 17:28:07.187041 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 17:28:07.187041 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 17:28:07.187041 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 17:28:07.188133 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:28:07.188133 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:28:07.188133 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 17:28:07.188133 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:28:07.188133 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 17:28:07.188133 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:28:07.188133 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:28:07.188133 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:28:07.188133 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:28:07.188133 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:28:07.188133 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:28:07.188133 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application with Lock( Aug 24 17:28:07.188133 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:28:07.188133 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:28:07.188133 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:28:07.188133 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:28:07.188133 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:28:07.188133 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:28:07.189322 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:28:07.189322 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:28:07.189322 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 17:28:07.189322 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 17:28:07.189322 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 17:28:07.189322 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 17:28:07.189322 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 17:28:07.189322 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 17:28:07.189322 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 17:28:07.189322 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Aug 24 17:28:07.189322 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 335/657] 10.222.0.45 () {56 vars in 1102 bytes} [Wed Aug 24 17:28:07 2022] GET /identity/v3/users/neutron => generated 85 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:28:07.189322 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5a449a-c6f9-4b24-85b3-0fbb2bb3f4b8 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:07.199401 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application [None req-120aa9b9-a762-4d2d-954c-56da16abe0b4 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 17:28:07.199401 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:28:07.199401 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:28:07.199401 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:28:07.199401 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:28:07.199401 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:28:07.199401 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:28:07.199401 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:28:07.199401 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:28:07.199401 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:28:07.199401 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:28:07.199401 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:28:07.199401 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 17:28:07.199401 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 17:28:07.199401 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 17:28:07.199401 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 17:28:07.200299 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:28:07.200299 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:28:07.200299 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 17:28:07.200299 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:28:07.200299 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 17:28:07.200299 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:28:07.200299 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:28:07.200299 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:28:07.200299 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:28:07.200299 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:28:07.200299 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:28:07.200299 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application with Lock( Aug 24 17:28:07.200299 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:28:07.200299 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:28:07.200299 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:28:07.200299 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:28:07.200299 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:28:07.200299 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:28:07.201699 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:28:07.201699 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:28:07.201699 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 17:28:07.201699 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 17:28:07.201699 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 17:28:07.201699 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 17:28:07.201699 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 17:28:07.201699 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 17:28:07.201699 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 17:28:07.201699 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Aug 24 17:28:07.201699 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 323/658] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Aug 24 17:28:07 2022] GET /identity/v3/users/cinder => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:28:07.201699 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86f7574-37b0-4491-986b-248f0a509c94 None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:07.202473 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86f7574-37b0-4491-986b-248f0a509c94 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:07.202473 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86f7574-37b0-4491-986b-248f0a509c94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:07.219846 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-a86f7574-37b0-4491-986b-248f0a509c94 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:28:07.221067 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-a86f7574-37b0-4491-986b-248f0a509c94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:28:07.285754 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 324/659] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:28:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:28:07.286498 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86f7574-37b0-4491-986b-248f0a509c94 None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:07.291064 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5a449a-c6f9-4b24-85b3-0fbb2bb3f4b8 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:07.332217 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b706cd-b6cd-42dd-bd3c-28536e93ed5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:07.332859 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b706cd-b6cd-42dd-bd3c-28536e93ed5d None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:07.333149 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b706cd-b6cd-42dd-bd3c-28536e93ed5d None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:07.333422 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b706cd-b6cd-42dd-bd3c-28536e93ed5d None admin] PATH_INFO: `/v3/projects` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:07.334403 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b706cd-b6cd-42dd-bd3c-28536e93ed5d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:07.334403 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b706cd-b6cd-42dd-bd3c-28536e93ed5d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:07.335867 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b706cd-b6cd-42dd-bd3c-28536e93ed5d None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:07.352603 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 336/660] 10.222.0.45 () {56 vars in 1148 bytes} [Wed Aug 24 17:28:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:07.352603 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b706cd-b6cd-42dd-bd3c-28536e93ed5d None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:07.369245 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:07.391422 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:07.397919 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86f7574-37b0-4491-986b-248f0a509c94 None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:07.433428 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8518a0-edf5-4f13-beb0-4e12eecec285 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:07.434066 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8518a0-edf5-4f13-beb0-4e12eecec285 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:07.434347 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8518a0-edf5-4f13-beb0-4e12eecec285 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:07.434621 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8518a0-edf5-4f13-beb0-4e12eecec285 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:07.441586 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8518a0-edf5-4f13-beb0-4e12eecec285 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:07.443163 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8518a0-edf5-4f13-beb0-4e12eecec285 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:07.454160 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b706cd-b6cd-42dd-bd3c-28536e93ed5d None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:07.455568 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application [None req-0f8518a0-edf5-4f13-beb0-4e12eecec285 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:28:07.455568 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:28:07.455568 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:28:07.455568 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:28:07.455568 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:28:07.455568 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:28:07.455568 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:28:07.455568 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:28:07.455568 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:28:07.455568 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:28:07.455568 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:28:07.455568 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:28:07.455568 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 17:28:07.455568 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 17:28:07.455568 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 17:28:07.456410 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 17:28:07.456410 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:28:07.456410 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:28:07.456410 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:28:07.456410 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:28:07.456410 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:28:07.456410 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:28:07.456410 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:28:07.456410 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application with Lock( Aug 24 17:28:07.456410 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:28:07.456410 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:28:07.456410 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:28:07.456410 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:28:07.456410 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:28:07.456410 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:28:07.456410 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:28:07.456410 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:28:07.457538 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 17:28:07.457538 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 17:28:07.457538 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 17:28:07.457538 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 17:28:07.457538 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 17:28:07.457538 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 17:28:07.457538 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:28:07.457538 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Aug 24 17:28:07.457538 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 325/661] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Aug 24 17:28:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:28:07.458050 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8518a0-edf5-4f13-beb0-4e12eecec285 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:07.493171 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d5cf5d-29cb-4fa4-b15f-04c2c4309918 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:07.493776 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d5cf5d-29cb-4fa4-b15f-04c2c4309918 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:07.494054 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d5cf5d-29cb-4fa4-b15f-04c2c4309918 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:07.494332 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d5cf5d-29cb-4fa4-b15f-04c2c4309918 None admin] PATH_INFO: `/v3/users` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:07.495566 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d5cf5d-29cb-4fa4-b15f-04c2c4309918 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:07.495868 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d5cf5d-29cb-4fa4-b15f-04c2c4309918 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:07.497308 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d5cf5d-29cb-4fa4-b15f-04c2c4309918 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:07.514411 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 337/662] 10.222.0.45 () {56 vars in 1139 bytes} [Wed Aug 24 17:28:07 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:07.515189 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d5cf5d-29cb-4fa4-b15f-04c2c4309918 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:07.567547 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8518a0-edf5-4f13-beb0-4e12eecec285 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:07.589162 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:07.600292 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cabf94-c4aa-4fec-b8de-cdf7aed42564 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:07.600922 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cabf94-c4aa-4fec-b8de-cdf7aed42564 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:07.601196 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cabf94-c4aa-4fec-b8de-cdf7aed42564 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:07.601462 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cabf94-c4aa-4fec-b8de-cdf7aed42564 None admin] PATH_INFO: `/v3/users` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:07.602192 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cabf94-c4aa-4fec-b8de-cdf7aed42564 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:07.602491 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cabf94-c4aa-4fec-b8de-cdf7aed42564 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:07.604440 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cabf94-c4aa-4fec-b8de-cdf7aed42564 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:07.614285 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d5cf5d-29cb-4fa4-b15f-04c2c4309918 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:07.621773 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 326/663] 10.222.0.45 () {56 vars in 1137 bytes} [Wed Aug 24 17:28:07 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:07.622570 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cabf94-c4aa-4fec-b8de-cdf7aed42564 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:07.637631 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:07.652472 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9a908b-0c39-46fe-b2a2-b88aad64ec2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:07.653128 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9a908b-0c39-46fe-b2a2-b88aad64ec2e None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:07.653460 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9a908b-0c39-46fe-b2a2-b88aad64ec2e None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:07.653740 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9a908b-0c39-46fe-b2a2-b88aad64ec2e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:07.654464 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9a908b-0c39-46fe-b2a2-b88aad64ec2e None admin] RBAC: Adding query filter params (role.id=ad6a81fb5d1941888577eb0ce9474882, scope.project.id=2ad8a32bd9d2434097bd08b111914ca1, user.id=df5fee57539447428dd1c72eab3cfd9d) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:07.654801 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9a908b-0c39-46fe-b2a2-b88aad64ec2e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:07.656221 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9a908b-0c39-46fe-b2a2-b88aad64ec2e None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:07.662052 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 338/664] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Aug 24 17:28:07 2022] GET /identity/v3/role_assignments?user.id=df5fee57539447428dd1c72eab3cfd9d&scope.project.id=2ad8a32bd9d2434097bd08b111914ca1&role.id=ad6a81fb5d1941888577eb0ce9474882 => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:07.662657 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9a908b-0c39-46fe-b2a2-b88aad64ec2e None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:07.733236 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cabf94-c4aa-4fec-b8de-cdf7aed42564 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:07.771784 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f745e6b0-9a49-4f23-a7bf-5c3960973e5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:07.772540 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f745e6b0-9a49-4f23-a7bf-5c3960973e5e None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:07.772983 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f745e6b0-9a49-4f23-a7bf-5c3960973e5e None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:07.773330 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f745e6b0-9a49-4f23-a7bf-5c3960973e5e None admin] PATH_INFO: `/v3/projects` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:07.774127 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f745e6b0-9a49-4f23-a7bf-5c3960973e5e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:07.774510 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f745e6b0-9a49-4f23-a7bf-5c3960973e5e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:07.776098 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f745e6b0-9a49-4f23-a7bf-5c3960973e5e None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:07.784019 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9a908b-0c39-46fe-b2a2-b88aad64ec2e None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:07.795126 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 327/665] 10.222.0.45 () {56 vars in 1148 bytes} [Wed Aug 24 17:28:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:07.795126 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f745e6b0-9a49-4f23-a7bf-5c3960973e5e None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:07.830398 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48831bc4-a788-4054-9f64-c82261ebccac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:07.831091 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48831bc4-a788-4054-9f64-c82261ebccac None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:07.831479 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48831bc4-a788-4054-9f64-c82261ebccac None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:07.831852 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48831bc4-a788-4054-9f64-c82261ebccac None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:07.840929 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:07.883081 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:07.908700 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f745e6b0-9a49-4f23-a7bf-5c3960973e5e None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:07.909295 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48831bc4-a788-4054-9f64-c82261ebccac None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:07.911892 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48831bc4-a788-4054-9f64-c82261ebccac None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:07.947006 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10dea45d-190f-4b48-b549-ee5770fd65c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:07.947833 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dea45d-190f-4b48-b549-ee5770fd65c1 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:07.948212 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dea45d-190f-4b48-b549-ee5770fd65c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:07.948615 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dea45d-190f-4b48-b549-ee5770fd65c1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:07.959718 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10dea45d-190f-4b48-b549-ee5770fd65c1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:28:07.966368 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application [None req-10dea45d-190f-4b48-b549-ee5770fd65c1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 17:28:07.966368 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:28:07.966368 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:28:07.966368 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:28:07.966368 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:28:07.966368 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:28:07.966368 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:28:07.966368 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:28:07.966368 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:28:07.966368 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:28:07.966368 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:28:07.966368 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:28:07.966368 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:28:07.966368 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:28:07.966368 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:28:07.966368 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:28:07.967486 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:28:07.967486 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:28:07.967486 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:28:07.967486 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:28:07.967486 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:28:07.967486 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:28:07.967486 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:28:07.967486 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:28:07.967486 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:28:07.967486 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:28:07.967486 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:28:07.967486 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application with Lock( Aug 24 17:28:07.967486 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:28:07.967486 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:28:07.967486 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:28:07.967486 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:28:07.973678 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:28:07.973678 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:28:07.973678 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:28:07.973678 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:28:07.973678 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:28:07.973678 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:28:07.973678 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:28:07.973678 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:28:07.973678 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:28:07.973678 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:28:07.973678 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 17:28:07.973678 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Aug 24 17:28:07.973678 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 328/666] 10.222.0.45 () {56 vars in 1102 bytes} [Wed Aug 24 17:28:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:28:07.973678 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10dea45d-190f-4b48-b549-ee5770fd65c1 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:08.028957 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 339/667] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Aug 24 17:28:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 17:28:08.029814 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48831bc4-a788-4054-9f64-c82261ebccac None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:08.070555 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:08.083323 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10dea45d-190f-4b48-b549-ee5770fd65c1 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:08.120329 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0960fff1-8977-4e16-a0fc-5c4ad99d411b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:08.121304 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0960fff1-8977-4e16-a0fc-5c4ad99d411b None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:08.121764 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0960fff1-8977-4e16-a0fc-5c4ad99d411b None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:08.122225 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0960fff1-8977-4e16-a0fc-5c4ad99d411b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:08.125553 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:08.131694 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48831bc4-a788-4054-9f64-c82261ebccac None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:08.169514 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3debe831-1298-4c0e-bc96-1283bea2f347 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:08.170119 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3debe831-1298-4c0e-bc96-1283bea2f347 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:08.171010 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3debe831-1298-4c0e-bc96-1283bea2f347 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:08.171010 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3debe831-1298-4c0e-bc96-1283bea2f347 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:08.171372 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3debe831-1298-4c0e-bc96-1283bea2f347 None admin] RBAC: Adding query filter params (role.id=ad6a81fb5d1941888577eb0ce9474882, scope.project.id=2ad8a32bd9d2434097bd08b111914ca1, user.id=053b7ce6d6f649ed91a26e434dab805e) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:08.171906 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3debe831-1298-4c0e-bc96-1283bea2f347 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:08.173146 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3debe831-1298-4c0e-bc96-1283bea2f347 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:08.179049 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 340/668] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Aug 24 17:28:08 2022] GET /identity/v3/role_assignments?user.id=053b7ce6d6f649ed91a26e434dab805e&scope.project.id=2ad8a32bd9d2434097bd08b111914ca1&role.id=ad6a81fb5d1941888577eb0ce9474882 => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:08.181934 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3debe831-1298-4c0e-bc96-1283bea2f347 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:08.201964 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0960fff1-8977-4e16-a0fc-5c4ad99d411b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:08.203518 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0960fff1-8977-4e16-a0fc-5c4ad99d411b None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:08.309819 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3debe831-1298-4c0e-bc96-1283bea2f347 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:08.311817 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:08.355232 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 329/669] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Aug 24 17:28:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 17:28:08.356037 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0960fff1-8977-4e16-a0fc-5c4ad99d411b None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:08.360302 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1c4b32-f28f-40c3-b05b-3192ad658db0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:08.360987 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1c4b32-f28f-40c3-b05b-3192ad658db0 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:08.361272 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1c4b32-f28f-40c3-b05b-3192ad658db0 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:08.362453 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1c4b32-f28f-40c3-b05b-3192ad658db0 None admin] PATH_INFO: `/v3/roles` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:08.363136 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1c4b32-f28f-40c3-b05b-3192ad658db0 None admin] RBAC: Adding query filter params (name=service) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:08.363444 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1c4b32-f28f-40c3-b05b-3192ad658db0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:08.364855 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1c4b32-f28f-40c3-b05b-3192ad658db0 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:08.374508 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 341/670] 10.222.0.45 () {56 vars in 1103 bytes} [Wed Aug 24 17:28:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:08.375329 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1c4b32-f28f-40c3-b05b-3192ad658db0 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:08.400360 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:08.496242 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0960fff1-8977-4e16-a0fc-5c4ad99d411b None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:08.502428 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1c4b32-f28f-40c3-b05b-3192ad658db0 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:08.533602 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dfd2ad-d2b6-4c1c-8719-0882a62c4c0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:08.534240 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dfd2ad-d2b6-4c1c-8719-0882a62c4c0c None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:08.534520 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dfd2ad-d2b6-4c1c-8719-0882a62c4c0c None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:08.534793 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dfd2ad-d2b6-4c1c-8719-0882a62c4c0c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:08.546725 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95dfd2ad-d2b6-4c1c-8719-0882a62c4c0c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:08.548379 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95dfd2ad-d2b6-4c1c-8719-0882a62c4c0c None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:08.548925 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b93984b-16cd-4759-8149-6abef2cd0ebb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:08.549804 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b93984b-16cd-4759-8149-6abef2cd0ebb None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:08.550500 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b93984b-16cd-4759-8149-6abef2cd0ebb None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:08.551351 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b93984b-16cd-4759-8149-6abef2cd0ebb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:08.563326 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application [None req-95dfd2ad-d2b6-4c1c-8719-0882a62c4c0c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:28:08.563326 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:28:08.563326 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:28:08.563326 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:28:08.563326 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:28:08.563326 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:28:08.563326 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:28:08.563326 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:28:08.563326 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:28:08.563326 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:28:08.563326 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:28:08.563326 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:28:08.563326 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:28:08.563326 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:28:08.563326 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:28:08.563326 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:28:08.564278 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:28:08.564278 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:28:08.564278 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:28:08.564278 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:28:08.564278 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:28:08.564278 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:28:08.564278 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:28:08.564278 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application with Lock( Aug 24 17:28:08.564278 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:28:08.564278 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:28:08.564278 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:28:08.564278 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:28:08.564278 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:28:08.564278 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:28:08.564278 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:28:08.564278 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:28:08.564278 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:28:08.565191 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:28:08.565191 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:28:08.565191 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Aug 24 17:28:08.565191 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 330/671] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 17:28:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:28:08.566373 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b93984b-16cd-4759-8149-6abef2cd0ebb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:08.568013 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b93984b-16cd-4759-8149-6abef2cd0ebb None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:08.568848 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dfd2ad-d2b6-4c1c-8719-0882a62c4c0c None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:08.587723 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application [None req-7b93984b-16cd-4759-8149-6abef2cd0ebb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:28:08.587723 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:28:08.587723 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:28:08.587723 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:28:08.587723 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:28:08.587723 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:28:08.587723 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:28:08.587723 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:28:08.587723 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:28:08.587723 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:28:08.587723 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:28:08.587723 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:28:08.587723 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:28:08.587723 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:28:08.587723 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:28:08.587723 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:28:08.589393 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:28:08.589393 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:28:08.589393 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:28:08.589393 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:28:08.589393 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:28:08.589393 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:28:08.589393 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:28:08.589393 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application with Lock( Aug 24 17:28:08.589393 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:28:08.589393 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:28:08.589393 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:28:08.589393 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:28:08.589393 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:28:08.589393 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:28:08.589393 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:28:08.589393 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:28:08.589393 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:28:08.590261 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:28:08.590261 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:28:08.590261 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Aug 24 17:28:08.590440 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 342/672] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 17:28:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:28:08.595846 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b93984b-16cd-4759-8149-6abef2cd0ebb None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:08.618522 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:08.687376 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:08.709745 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dfd2ad-d2b6-4c1c-8719-0882a62c4c0c None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:08.709932 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b93984b-16cd-4759-8149-6abef2cd0ebb None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:08.746921 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22338c7-50cc-4b95-b8de-0bee3f1bab8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:08.747530 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22338c7-50cc-4b95-b8de-0bee3f1bab8d None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:08.747810 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22338c7-50cc-4b95-b8de-0bee3f1bab8d None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:08.748085 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22338c7-50cc-4b95-b8de-0bee3f1bab8d None admin] PATH_INFO: `/v3/domains` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:08.748763 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22338c7-50cc-4b95-b8de-0bee3f1bab8d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:08.749066 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22338c7-50cc-4b95-b8de-0bee3f1bab8d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:08.750522 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22338c7-50cc-4b95-b8de-0bee3f1bab8d None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:08.760836 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f7c004-a25b-4f41-99c3-b8326395839f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:08.761506 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f7c004-a25b-4f41-99c3-b8326395839f None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:08.761784 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f7c004-a25b-4f41-99c3-b8326395839f None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:08.762056 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f7c004-a25b-4f41-99c3-b8326395839f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:08.767246 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 343/673] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 24 17:28:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:08.768345 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22338c7-50cc-4b95-b8de-0bee3f1bab8d None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:08.781909 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f7c004-a25b-4f41-99c3-b8326395839f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:08.784161 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f7c004-a25b-4f41-99c3-b8326395839f None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:08.803185 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application [None req-d6f7c004-a25b-4f41-99c3-b8326395839f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:28:08.803185 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:28:08.803185 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:28:08.803185 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:28:08.803185 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:28:08.803185 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:28:08.803185 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:28:08.803185 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:28:08.803185 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:28:08.803185 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:28:08.803185 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:28:08.803185 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:28:08.803185 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:28:08.803185 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:28:08.803185 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:28:08.803185 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:28:08.804129 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:28:08.804129 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:28:08.804129 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:28:08.804129 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:28:08.804129 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:28:08.804129 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:28:08.804129 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:28:08.804129 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application with Lock( Aug 24 17:28:08.804129 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:28:08.804129 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:28:08.804129 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:28:08.804129 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:28:08.804129 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:28:08.804129 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:28:08.804129 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:28:08.804129 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:28:08.804129 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:28:08.805048 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:28:08.805048 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:28:08.805048 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Aug 24 17:28:08.812946 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 331/674] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 17:28:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:28:08.814379 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f7c004-a25b-4f41-99c3-b8326395839f None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:08.865153 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:08.896054 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22338c7-50cc-4b95-b8de-0bee3f1bab8d None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:08.930850 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f7c004-a25b-4f41-99c3-b8326395839f None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:08.943931 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586f3965-3104-4e0b-aef7-3a331f1cbaf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:08.944890 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586f3965-3104-4e0b-aef7-3a331f1cbaf4 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:08.945177 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586f3965-3104-4e0b-aef7-3a331f1cbaf4 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:08.945452 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586f3965-3104-4e0b-aef7-3a331f1cbaf4 None admin] PATH_INFO: `/v3/domains` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:08.946108 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586f3965-3104-4e0b-aef7-3a331f1cbaf4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:08.946414 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586f3965-3104-4e0b-aef7-3a331f1cbaf4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:08.948530 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586f3965-3104-4e0b-aef7-3a331f1cbaf4 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:08.964701 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 344/675] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 24 17:28:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:08.965908 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586f3965-3104-4e0b-aef7-3a331f1cbaf4 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:08.975432 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f562f487-f32a-4ee7-aafe-0ca0ba752cde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:08.976328 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f562f487-f32a-4ee7-aafe-0ca0ba752cde None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:08.976864 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f562f487-f32a-4ee7-aafe-0ca0ba752cde None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:08.977330 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f562f487-f32a-4ee7-aafe-0ca0ba752cde None admin] PATH_INFO: `/v3/projects/service` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:08.985832 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f562f487-f32a-4ee7-aafe-0ca0ba752cde None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:08.987714 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f562f487-f32a-4ee7-aafe-0ca0ba752cde None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:08.989841 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:09.002470 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application [None req-f562f487-f32a-4ee7-aafe-0ca0ba752cde None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:28:09.002470 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:28:09.002470 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:28:09.002470 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:28:09.002470 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:28:09.002470 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:28:09.002470 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:28:09.002470 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:28:09.002470 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:28:09.002470 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:28:09.002470 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:28:09.002470 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:28:09.002470 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 17:28:09.002470 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 17:28:09.002470 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 17:28:09.003464 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 17:28:09.003464 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:28:09.003464 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:28:09.003464 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:28:09.003464 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:28:09.003464 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:28:09.003464 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:28:09.003464 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:28:09.003464 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application with Lock( Aug 24 17:28:09.003464 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:28:09.003464 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:28:09.003464 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:28:09.003464 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:28:09.003464 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:28:09.003464 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:28:09.003464 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:28:09.003464 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:28:09.004359 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 17:28:09.004359 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 17:28:09.004359 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 17:28:09.004359 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 17:28:09.004359 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 17:28:09.004359 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 17:28:09.004359 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:28:09.004359 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Aug 24 17:28:09.006921 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 332/676] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Aug 24 17:28:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:28:09.008092 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f562f487-f32a-4ee7-aafe-0ca0ba752cde None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:09.090204 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586f3965-3104-4e0b-aef7-3a331f1cbaf4 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:09.116416 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f562f487-f32a-4ee7-aafe-0ca0ba752cde None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:09.131260 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e08f14-c5b0-4c82-9d4d-1dd90ac52fe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:09.131877 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e08f14-c5b0-4c82-9d4d-1dd90ac52fe6 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:09.132184 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e08f14-c5b0-4c82-9d4d-1dd90ac52fe6 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:09.132835 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e08f14-c5b0-4c82-9d4d-1dd90ac52fe6 None admin] PATH_INFO: `/v3/domains` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:09.133630 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e08f14-c5b0-4c82-9d4d-1dd90ac52fe6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:09.134046 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e08f14-c5b0-4c82-9d4d-1dd90ac52fe6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:09.136104 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e08f14-c5b0-4c82-9d4d-1dd90ac52fe6 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:09.138701 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:09.153206 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 345/677] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 24 17:28:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:09.154370 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e08f14-c5b0-4c82-9d4d-1dd90ac52fe6 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:09.170203 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ef0d6c-6ab0-4464-8fa4-ba1c004de324 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:09.170854 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ef0d6c-6ab0-4464-8fa4-ba1c004de324 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:09.171134 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ef0d6c-6ab0-4464-8fa4-ba1c004de324 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:09.171409 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ef0d6c-6ab0-4464-8fa4-ba1c004de324 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:09.180312 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ef0d6c-6ab0-4464-8fa4-ba1c004de324 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:09.181795 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ef0d6c-6ab0-4464-8fa4-ba1c004de324 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:09.197142 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application [None req-05ef0d6c-6ab0-4464-8fa4-ba1c004de324 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:28:09.197142 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:28:09.197142 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:28:09.197142 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:28:09.197142 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:28:09.197142 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:28:09.197142 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:28:09.197142 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:28:09.197142 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:28:09.197142 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:28:09.197142 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:28:09.197142 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:28:09.197142 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 17:28:09.197142 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 17:28:09.197142 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 17:28:09.198618 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 17:28:09.198618 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:28:09.198618 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:28:09.198618 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:28:09.198618 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:28:09.198618 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:28:09.198618 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:28:09.198618 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:28:09.198618 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application with Lock( Aug 24 17:28:09.198618 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:28:09.198618 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:28:09.198618 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:28:09.198618 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:28:09.198618 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:28:09.198618 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:28:09.198618 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:28:09.198618 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:28:09.199482 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 17:28:09.199482 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 17:28:09.199482 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 17:28:09.199482 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 17:28:09.199482 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 17:28:09.199482 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 17:28:09.199482 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:28:09.199482 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Aug 24 17:28:09.203556 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 333/678] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Aug 24 17:28:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:28:09.204439 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ef0d6c-6ab0-4464-8fa4-ba1c004de324 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:09.271921 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e08f14-c5b0-4c82-9d4d-1dd90ac52fe6 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:09.282248 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:09.310014 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bc4446-2d7e-49ec-a83e-5a81157ac287 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:09.310631 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bc4446-2d7e-49ec-a83e-5a81157ac287 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:09.310911 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bc4446-2d7e-49ec-a83e-5a81157ac287 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:09.311189 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bc4446-2d7e-49ec-a83e-5a81157ac287 None admin] PATH_INFO: `/v3/projects` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:09.311891 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bc4446-2d7e-49ec-a83e-5a81157ac287 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:09.312197 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bc4446-2d7e-49ec-a83e-5a81157ac287 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:09.313665 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bc4446-2d7e-49ec-a83e-5a81157ac287 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:09.331009 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 346/679] 10.222.0.45 () {56 vars in 1148 bytes} [Wed Aug 24 17:28:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:09.331955 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bc4446-2d7e-49ec-a83e-5a81157ac287 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:09.332544 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ef0d6c-6ab0-4464-8fa4-ba1c004de324 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:09.376425 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c3eabe-bd6e-48d0-8093-75e5a5821048 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:09.377094 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c3eabe-bd6e-48d0-8093-75e5a5821048 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:09.377375 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c3eabe-bd6e-48d0-8093-75e5a5821048 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:09.377807 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c3eabe-bd6e-48d0-8093-75e5a5821048 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:09.388055 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c3eabe-bd6e-48d0-8093-75e5a5821048 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:09.388620 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:09.389892 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c3eabe-bd6e-48d0-8093-75e5a5821048 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:09.403862 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application [None req-f4c3eabe-bd6e-48d0-8093-75e5a5821048 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 17:28:09.403862 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:28:09.403862 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:28:09.403862 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:28:09.403862 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:28:09.403862 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:28:09.403862 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:28:09.403862 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:28:09.403862 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:28:09.403862 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:28:09.403862 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:28:09.403862 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:28:09.403862 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 17:28:09.403862 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 17:28:09.403862 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 17:28:09.403862 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 17:28:09.404897 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:28:09.404897 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:28:09.404897 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 17:28:09.404897 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:28:09.404897 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 17:28:09.404897 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:28:09.404897 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:28:09.404897 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:28:09.404897 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:28:09.404897 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:28:09.404897 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:28:09.404897 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application with Lock( Aug 24 17:28:09.404897 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:28:09.404897 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:28:09.404897 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:28:09.404897 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:28:09.404897 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:28:09.404897 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:28:09.405797 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:28:09.405797 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:28:09.405797 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 17:28:09.405797 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 17:28:09.405797 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 17:28:09.405797 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 17:28:09.405797 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 17:28:09.405797 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 17:28:09.405797 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 17:28:09.405797 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Aug 24 17:28:09.409846 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 334/680] 10.222.0.45 () {56 vars in 1096 bytes} [Wed Aug 24 17:28:09 2022] GET /identity/v3/users/swift => generated 83 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:28:09.410693 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c3eabe-bd6e-48d0-8093-75e5a5821048 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:09.450624 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bc4446-2d7e-49ec-a83e-5a81157ac287 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:09.495350 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e575e85b-a49a-47e6-ad13-a8c3fe478e1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:09.495967 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e575e85b-a49a-47e6-ad13-a8c3fe478e1c None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:09.496498 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e575e85b-a49a-47e6-ad13-a8c3fe478e1c None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:09.496816 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e575e85b-a49a-47e6-ad13-a8c3fe478e1c None admin] PATH_INFO: `/v3/projects` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:09.497994 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e575e85b-a49a-47e6-ad13-a8c3fe478e1c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:09.498304 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e575e85b-a49a-47e6-ad13-a8c3fe478e1c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:09.500332 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e575e85b-a49a-47e6-ad13-a8c3fe478e1c None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:09.516727 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c3eabe-bd6e-48d0-8093-75e5a5821048 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:09.520647 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 347/681] 10.222.0.45 () {56 vars in 1148 bytes} [Wed Aug 24 17:28:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:09.521568 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e575e85b-a49a-47e6-ad13-a8c3fe478e1c None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:09.542767 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:09.567550 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f82a6d-9fd3-4454-96a8-970b989d53b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:09.568590 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f82a6d-9fd3-4454-96a8-970b989d53b6 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:09.569404 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f82a6d-9fd3-4454-96a8-970b989d53b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:09.569948 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f82a6d-9fd3-4454-96a8-970b989d53b6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:09.570923 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f82a6d-9fd3-4454-96a8-970b989d53b6 None admin] RBAC: Adding query filter params (role.id=ad6a81fb5d1941888577eb0ce9474882, scope.project.id=2ad8a32bd9d2434097bd08b111914ca1, user.id=43ad98a250a144099ccd18378aedd4b5) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:09.571534 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f82a6d-9fd3-4454-96a8-970b989d53b6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:09.573983 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f82a6d-9fd3-4454-96a8-970b989d53b6 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:09.582408 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 335/682] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Aug 24 17:28:09 2022] GET /identity/v3/role_assignments?user.id=43ad98a250a144099ccd18378aedd4b5&scope.project.id=2ad8a32bd9d2434097bd08b111914ca1&role.id=ad6a81fb5d1941888577eb0ce9474882 => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:09.583365 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f82a6d-9fd3-4454-96a8-970b989d53b6 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:09.679252 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e575e85b-a49a-47e6-ad13-a8c3fe478e1c None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:09.699188 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:09.714028 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f82a6d-9fd3-4454-96a8-970b989d53b6 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:09.726942 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c674f364-3209-4a54-8568-79df959a10c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:09.727646 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c674f364-3209-4a54-8568-79df959a10c3 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:09.727877 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c674f364-3209-4a54-8568-79df959a10c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:09.728153 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c674f364-3209-4a54-8568-79df959a10c3 None admin] PATH_INFO: `/v3/users` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:09.729017 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c674f364-3209-4a54-8568-79df959a10c3 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:09.729291 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c674f364-3209-4a54-8568-79df959a10c3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:09.731026 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c674f364-3209-4a54-8568-79df959a10c3 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:09.757859 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f74c9b-e93d-4a8c-a76e-b08d8bf35fb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:09.758741 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f74c9b-e93d-4a8c-a76e-b08d8bf35fb3 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:09.759177 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f74c9b-e93d-4a8c-a76e-b08d8bf35fb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:09.759449 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f74c9b-e93d-4a8c-a76e-b08d8bf35fb3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:09.760260 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f74c9b-e93d-4a8c-a76e-b08d8bf35fb3 None admin] RBAC: Adding query filter params (role.id=ad6a81fb5d1941888577eb0ce9474882, scope.project.id=2ad8a32bd9d2434097bd08b111914ca1, user.id=59be342541264942ad87f60ff4de6c8e) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:09.761073 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f74c9b-e93d-4a8c-a76e-b08d8bf35fb3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:09.762505 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f74c9b-e93d-4a8c-a76e-b08d8bf35fb3 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:09.764115 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 348/683] 10.222.0.45 () {56 vars in 1135 bytes} [Wed Aug 24 17:28:09 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:09.765351 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695bccc0-a054-4fc8-a527-7651793c0e3e None None] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:09.766077 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695bccc0-a054-4fc8-a527-7651793c0e3e None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:09.766077 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695bccc0-a054-4fc8-a527-7651793c0e3e None None] PATH_INFO: `/` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:09.767301 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 349/684] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:28:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:28:09.772415 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2292fd88-5f53-4c54-9611-e82186fbff19 None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:09.772769 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2292fd88-5f53-4c54-9611-e82186fbff19 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:09.774887 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 336/685] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Aug 24 17:28:09 2022] GET /identity/v3/role_assignments?user.id=59be342541264942ad87f60ff4de6c8e&scope.project.id=2ad8a32bd9d2434097bd08b111914ca1&role.id=ad6a81fb5d1941888577eb0ce9474882 => generated 608 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:09.775959 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f74c9b-e93d-4a8c-a76e-b08d8bf35fb3 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:09.776178 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2292fd88-5f53-4c54-9611-e82186fbff19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:09.799292 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-2292fd88-5f53-4c54-9611-e82186fbff19 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:28:09.801615 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-2292fd88-5f53-4c54-9611-e82186fbff19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:28:09.838422 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:09.893455 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 350/686] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:28:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:28:09.909174 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f74c9b-e93d-4a8c-a76e-b08d8bf35fb3 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:09.948010 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c7bb90-0c2e-4b22-bd08-0c07d3d5705f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:09.948614 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c7bb90-0c2e-4b22-bd08-0c07d3d5705f None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:09.948901 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c7bb90-0c2e-4b22-bd08-0c07d3d5705f None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:09.949178 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c7bb90-0c2e-4b22-bd08-0c07d3d5705f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:10.021370 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c7bb90-0c2e-4b22-bd08-0c07d3d5705f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:10.022893 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c7bb90-0c2e-4b22-bd08-0c07d3d5705f None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:10.080933 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:10.146706 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 337/687] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Aug 24 17:28:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 17:28:10.154951 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2292fd88-5f53-4c54-9611-e82186fbff19 None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:10.200160 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c7bb90-0c2e-4b22-bd08-0c07d3d5705f None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:10.290560 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2292fd88-5f53-4c54-9611-e82186fbff19 None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:10.299711 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:10.328288 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b42b731-d182-4a80-9e96-98d38bd6c035 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:10.328971 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b42b731-d182-4a80-9e96-98d38bd6c035 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:10.329249 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b42b731-d182-4a80-9e96-98d38bd6c035 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:10.329519 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b42b731-d182-4a80-9e96-98d38bd6c035 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:10.340514 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b42b731-d182-4a80-9e96-98d38bd6c035 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:10.342009 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b42b731-d182-4a80-9e96-98d38bd6c035 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:10.358336 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application [None req-1b42b731-d182-4a80-9e96-98d38bd6c035 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:28:10.358336 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:28:10.358336 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:28:10.358336 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:28:10.358336 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:28:10.358336 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:28:10.358336 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:28:10.358336 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:28:10.358336 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:28:10.358336 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:28:10.358336 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:28:10.358336 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:28:10.358336 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:28:10.358336 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:28:10.358336 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:28:10.358336 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:28:10.359408 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:28:10.359408 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:28:10.359408 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:28:10.359408 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:28:10.359408 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:28:10.359408 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:28:10.359408 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:28:10.359408 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application with Lock( Aug 24 17:28:10.359408 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:28:10.359408 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:28:10.359408 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:28:10.359408 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:28:10.359408 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:28:10.359408 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:28:10.359408 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:28:10.359408 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:28:10.359408 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:28:10.361018 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:28:10.361018 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:28:10.361018 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Aug 24 17:28:10.361018 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 351/688] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 17:28:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:28:10.361018 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862e571c-4c38-422a-b9e0-986287e32dc2 None None] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:10.361520 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862e571c-4c38-422a-b9e0-986287e32dc2 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:10.362313 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862e571c-4c38-422a-b9e0-986287e32dc2 None None] PATH_INFO: `/` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:10.365933 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 352/689] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:28:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:28:10.365933 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862e571c-4c38-422a-b9e0-986287e32dc2 None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:10.391385 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c7bb90-0c2e-4b22-bd08-0c07d3d5705f None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:10.429229 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5a006d-a129-4c71-9e79-0a4d385efc69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:10.430233 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5a006d-a129-4c71-9e79-0a4d385efc69 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:10.430662 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5a006d-a129-4c71-9e79-0a4d385efc69 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:10.431082 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5a006d-a129-4c71-9e79-0a4d385efc69 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:10.442035 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5a006d-a129-4c71-9e79-0a4d385efc69 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:10.443842 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5a006d-a129-4c71-9e79-0a4d385efc69 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:10.451304 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:10.460052 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application [None req-8b5a006d-a129-4c71-9e79-0a4d385efc69 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:28:10.460052 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:28:10.460052 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:28:10.460052 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:28:10.460052 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:28:10.460052 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:28:10.460052 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:28:10.460052 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:28:10.460052 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:28:10.460052 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:28:10.460052 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:28:10.460052 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:28:10.460052 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:28:10.460052 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:28:10.460052 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:28:10.460052 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:28:10.461041 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:28:10.461041 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:28:10.461041 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:28:10.461041 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:28:10.461041 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:28:10.461041 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:28:10.461041 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:28:10.461041 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application with Lock( Aug 24 17:28:10.461041 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:28:10.461041 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:28:10.461041 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:28:10.461041 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:28:10.461041 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:28:10.461041 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:28:10.461041 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:28:10.461041 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:28:10.461041 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:28:10.461895 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:28:10.461895 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:28:10.461895 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Aug 24 17:28:10.464699 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 338/690] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 17:28:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:28:10.466110 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8b1d8b-54d1-447f-a7ec-834dc553c813 None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:10.466589 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8b1d8b-54d1-447f-a7ec-834dc553c813 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:10.467050 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8b1d8b-54d1-447f-a7ec-834dc553c813 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:10.486438 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-5c8b1d8b-54d1-447f-a7ec-834dc553c813 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:28:10.486953 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862e571c-4c38-422a-b9e0-986287e32dc2 None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:10.488217 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-5c8b1d8b-54d1-447f-a7ec-834dc553c813 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:28:10.528452 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c135e3-458c-4585-a931-14a9d55d4fb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:10.529496 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c135e3-458c-4585-a931-14a9d55d4fb0 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:10.530044 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c135e3-458c-4585-a931-14a9d55d4fb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:10.530529 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c135e3-458c-4585-a931-14a9d55d4fb0 None admin] PATH_INFO: `/v3/domains` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:10.531378 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c135e3-458c-4585-a931-14a9d55d4fb0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:10.531888 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c135e3-458c-4585-a931-14a9d55d4fb0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:10.533905 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c135e3-458c-4585-a931-14a9d55d4fb0 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:10.550661 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 353/691] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 24 17:28:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:10.552324 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c135e3-458c-4585-a931-14a9d55d4fb0 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:10.566662 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 339/692] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:28:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:28:10.566662 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8b1d8b-54d1-447f-a7ec-834dc553c813 None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:10.584192 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:10.736389 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c135e3-458c-4585-a931-14a9d55d4fb0 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:10.754040 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8b1d8b-54d1-447f-a7ec-834dc553c813 None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:10.785458 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:10.806210 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921d2037-99d1-4648-84e4-0913167d4d2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:10.807153 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921d2037-99d1-4648-84e4-0913167d4d2a None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:10.807634 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921d2037-99d1-4648-84e4-0913167d4d2a None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:10.808117 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921d2037-99d1-4648-84e4-0913167d4d2a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:10.815816 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921d2037-99d1-4648-84e4-0913167d4d2a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:10.817631 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921d2037-99d1-4648-84e4-0913167d4d2a None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:10.829433 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application [None req-921d2037-99d1-4648-84e4-0913167d4d2a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:28:10.829433 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:28:10.829433 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:28:10.829433 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:28:10.829433 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:28:10.829433 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:28:10.829433 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:28:10.829433 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:28:10.829433 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:28:10.829433 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:28:10.829433 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:28:10.829433 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:28:10.829433 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 17:28:10.829433 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 17:28:10.829433 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 17:28:10.830302 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 17:28:10.830302 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:28:10.830302 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:28:10.830302 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:28:10.830302 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:28:10.830302 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:28:10.830302 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:28:10.830302 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:28:10.830302 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application with Lock( Aug 24 17:28:10.830302 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:28:10.830302 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:28:10.830302 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:28:10.830302 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:28:10.830302 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:28:10.830302 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:28:10.830302 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:28:10.830302 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:28:10.831163 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 17:28:10.831163 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 17:28:10.831163 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 17:28:10.831163 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 17:28:10.831163 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 17:28:10.831163 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 17:28:10.831163 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:28:10.831163 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Aug 24 17:28:10.833571 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 340/693] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Aug 24 17:28:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:28:10.842581 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fabee7-d37b-412a-889c-a1e4e2264578 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:10.842581 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fabee7-d37b-412a-889c-a1e4e2264578 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:10.842581 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fabee7-d37b-412a-889c-a1e4e2264578 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:10.842581 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fabee7-d37b-412a-889c-a1e4e2264578 None admin] PATH_INFO: `/v3/domains` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:10.842924 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fabee7-d37b-412a-889c-a1e4e2264578 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:10.842924 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fabee7-d37b-412a-889c-a1e4e2264578 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:10.842924 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fabee7-d37b-412a-889c-a1e4e2264578 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:10.846756 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921d2037-99d1-4648-84e4-0913167d4d2a None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:10.854533 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 354/694] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 24 17:28:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:10.861480 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fabee7-d37b-412a-889c-a1e4e2264578 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:10.964784 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:10.971630 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921d2037-99d1-4648-84e4-0913167d4d2a None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:10.990553 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fabee7-d37b-412a-889c-a1e4e2264578 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:11.008650 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d950cd67-ec9c-4c0a-8ce0-00ae8c17c957 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:11.009769 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d950cd67-ec9c-4c0a-8ce0-00ae8c17c957 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:11.010970 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d950cd67-ec9c-4c0a-8ce0-00ae8c17c957 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:11.011472 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d950cd67-ec9c-4c0a-8ce0-00ae8c17c957 None admin] PATH_INFO: `/v3/projects` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:11.012410 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d950cd67-ec9c-4c0a-8ce0-00ae8c17c957 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:11.013065 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d950cd67-ec9c-4c0a-8ce0-00ae8c17c957 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:11.014843 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d950cd67-ec9c-4c0a-8ce0-00ae8c17c957 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:11.029124 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2630bdb-33f3-4e5a-924c-1e06a523804b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:11.029724 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2630bdb-33f3-4e5a-924c-1e06a523804b None admin] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:11.030060 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2630bdb-33f3-4e5a-924c-1e06a523804b None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:11.030269 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2630bdb-33f3-4e5a-924c-1e06a523804b None admin] PATH_INFO: `/v3/users` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:11.030731 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 341/695] 10.222.0.45 () {56 vars in 1148 bytes} [Wed Aug 24 17:28:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:28:11.030964 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2630bdb-33f3-4e5a-924c-1e06a523804b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:11.031995 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d950cd67-ec9c-4c0a-8ce0-00ae8c17c957 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:11.032405 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2630bdb-33f3-4e5a-924c-1e06a523804b None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:11.064845 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 355/696] 10.222.0.45 () {60 vars in 1132 bytes} [Wed Aug 24 17:28:10 2022] POST /identity/v3/users => generated 254 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:28:11.065925 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2630bdb-33f3-4e5a-924c-1e06a523804b None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:11.088503 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:11.147226 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d950cd67-ec9c-4c0a-8ce0-00ae8c17c957 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:11.186082 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a357124-4e31-4a1e-8e3d-e9719d9d38f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:11.187102 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a357124-4e31-4a1e-8e3d-e9719d9d38f1 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:11.187601 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a357124-4e31-4a1e-8e3d-e9719d9d38f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:11.188074 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a357124-4e31-4a1e-8e3d-e9719d9d38f1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:11.200367 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a357124-4e31-4a1e-8e3d-e9719d9d38f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:28:11.206400 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application [None req-5a357124-4e31-4a1e-8e3d-e9719d9d38f1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:28:11.206400 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:28:11.206400 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:28:11.206400 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:28:11.206400 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:28:11.206400 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:28:11.206400 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:28:11.206400 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:28:11.206400 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:28:11.206400 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:28:11.206400 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:28:11.206400 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:28:11.206400 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:28:11.206400 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:28:11.206400 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:28:11.206400 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:28:11.207339 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:28:11.207339 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:28:11.207339 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:28:11.207339 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:28:11.207339 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:28:11.207339 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:28:11.207339 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:28:11.207339 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:28:11.207339 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:28:11.207339 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:28:11.207339 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:28:11.207339 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application with Lock( Aug 24 17:28:11.207339 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:28:11.207339 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:28:11.207339 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:28:11.207339 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:28:11.208244 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:28:11.208244 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:28:11.208244 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:28:11.208244 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:28:11.208244 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:28:11.208244 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:28:11.208244 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:28:11.208244 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:28:11.208244 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:28:11.208244 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:28:11.208244 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:28:11.208244 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Aug 24 17:28:11.211144 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 342/697] 10.222.0.45 () {56 vars in 1096 bytes} [Wed Aug 24 17:28:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:28:11.221633 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a357124-4e31-4a1e-8e3d-e9719d9d38f1 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:11.243876 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2630bdb-33f3-4e5a-924c-1e06a523804b None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:11.279008 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:11.284258 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd15c31-c396-4224-8240-3b4312d8e417 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:11.284930 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd15c31-c396-4224-8240-3b4312d8e417 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:11.285199 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd15c31-c396-4224-8240-3b4312d8e417 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:11.285476 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd15c31-c396-4224-8240-3b4312d8e417 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:11.286240 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd15c31-c396-4224-8240-3b4312d8e417 None admin] RBAC: Adding query filter params (role.id=ad6a81fb5d1941888577eb0ce9474882, scope.project.id=2ad8a32bd9d2434097bd08b111914ca1, user.id=90f0772b64c84faaa689e7272fa97898) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:11.286538 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd15c31-c396-4224-8240-3b4312d8e417 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:11.288001 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd15c31-c396-4224-8240-3b4312d8e417 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:11.303865 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 356/698] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Aug 24 17:28:11 2022] GET /identity/v3/role_assignments?user.id=90f0772b64c84faaa689e7272fa97898&scope.project.id=2ad8a32bd9d2434097bd08b111914ca1&role.id=ad6a81fb5d1941888577eb0ce9474882 => generated 608 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:11.384107 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a357124-4e31-4a1e-8e3d-e9719d9d38f1 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:11.395292 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:11.423814 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5f8da0-2acd-44a2-88ae-60ee4ef9561b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:11.424442 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5f8da0-2acd-44a2-88ae-60ee4ef9561b None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:11.424756 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5f8da0-2acd-44a2-88ae-60ee4ef9561b None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:11.425028 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5f8da0-2acd-44a2-88ae-60ee4ef9561b None admin] PATH_INFO: `/v3/roles` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:11.425715 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5f8da0-2acd-44a2-88ae-60ee4ef9561b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:11.426016 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5f8da0-2acd-44a2-88ae-60ee4ef9561b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:11.427440 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5f8da0-2acd-44a2-88ae-60ee4ef9561b None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:11.437840 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 343/699] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Aug 24 17:28:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:28:11.443430 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd15c31-c396-4224-8240-3b4312d8e417 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:11.552535 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd15c31-c396-4224-8240-3b4312d8e417 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:11.593422 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98799bc6-3bea-4721-bbb2-bd461c53715f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:11.594391 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98799bc6-3bea-4721-bbb2-bd461c53715f None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:11.594671 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98799bc6-3bea-4721-bbb2-bd461c53715f None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:11.594940 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98799bc6-3bea-4721-bbb2-bd461c53715f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:11.609674 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98799bc6-3bea-4721-bbb2-bd461c53715f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:11.611519 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98799bc6-3bea-4721-bbb2-bd461c53715f None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:11.627694 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application [None req-98799bc6-3bea-4721-bbb2-bd461c53715f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:28:11.627694 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:28:11.627694 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:28:11.627694 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:28:11.627694 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:28:11.627694 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:28:11.627694 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:28:11.627694 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:28:11.627694 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:28:11.627694 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:28:11.627694 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:28:11.627694 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:28:11.627694 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:28:11.627694 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:28:11.627694 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:28:11.627694 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:28:11.628751 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:28:11.628751 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:28:11.628751 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:28:11.628751 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:28:11.628751 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:28:11.628751 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:28:11.628751 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:28:11.628751 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application with Lock( Aug 24 17:28:11.628751 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:28:11.628751 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:28:11.628751 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:28:11.628751 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:28:11.628751 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:28:11.628751 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:28:11.628751 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:28:11.628751 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:28:11.628751 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:28:11.629645 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:28:11.629645 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:28:11.629645 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Aug 24 17:28:11.631783 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 357/700] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 17:28:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:28:11.637231 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5f8da0-2acd-44a2-88ae-60ee4ef9561b None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:11.779343 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5f8da0-2acd-44a2-88ae-60ee4ef9561b None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:11.827392 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07e1cd7-f78f-4836-816a-2aacac62b6e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:11.828011 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07e1cd7-f78f-4836-816a-2aacac62b6e5 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:11.828239 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07e1cd7-f78f-4836-816a-2aacac62b6e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:11.828469 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07e1cd7-f78f-4836-816a-2aacac62b6e5 None admin] PATH_INFO: `/v3/domains` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:11.829143 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07e1cd7-f78f-4836-816a-2aacac62b6e5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:11.829445 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07e1cd7-f78f-4836-816a-2aacac62b6e5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:11.830832 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07e1cd7-f78f-4836-816a-2aacac62b6e5 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:11.851768 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 344/701] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 24 17:28:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:11.858035 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98799bc6-3bea-4721-bbb2-bd461c53715f None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:11.969767 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:11.991415 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253d0fba-58f9-402a-9837-8423d1aaa011 None None] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:11.991708 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253d0fba-58f9-402a-9837-8423d1aaa011 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:11.991963 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253d0fba-58f9-402a-9837-8423d1aaa011 None None] PATH_INFO: `/` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:11.994654 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 345/702] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:28:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:28:11.999004 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480b12c6-5b37-4921-bc71-de1d4ce9ea13 None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:11.999282 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480b12c6-5b37-4921-bc71-de1d4ce9ea13 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:11.999552 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480b12c6-5b37-4921-bc71-de1d4ce9ea13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:12.019533 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-480b12c6-5b37-4921-bc71-de1d4ce9ea13 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:28:12.020740 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-480b12c6-5b37-4921-bc71-de1d4ce9ea13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:28:12.056823 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:12.059711 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98799bc6-3bea-4721-bbb2-bd461c53715f None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:12.095859 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef224c7-be6e-4e42-8cdb-dcc9d10930ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:12.096443 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef224c7-be6e-4e42-8cdb-dcc9d10930ec None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:12.096755 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef224c7-be6e-4e42-8cdb-dcc9d10930ec None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:12.097033 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef224c7-be6e-4e42-8cdb-dcc9d10930ec None admin] PATH_INFO: `/v3/users/nova` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:12.104887 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef224c7-be6e-4e42-8cdb-dcc9d10930ec None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:12.106783 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef224c7-be6e-4e42-8cdb-dcc9d10930ec None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:12.116793 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 346/703] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:28:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 17:28:12.118253 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application [None req-2ef224c7-be6e-4e42-8cdb-dcc9d10930ec None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 17:28:12.118253 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:28:12.118253 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:28:12.118253 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:28:12.118253 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:28:12.118253 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:28:12.118253 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:28:12.118253 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:28:12.118253 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:28:12.118253 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:28:12.118253 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:28:12.118253 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:28:12.118253 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 17:28:12.118253 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 17:28:12.118253 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 17:28:12.118253 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 17:28:12.119240 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:28:12.119240 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:28:12.119240 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 17:28:12.119240 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:28:12.119240 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 17:28:12.119240 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:28:12.119240 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:28:12.119240 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:28:12.119240 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:28:12.119240 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:28:12.119240 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:28:12.119240 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application with Lock( Aug 24 17:28:12.119240 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:28:12.119240 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:28:12.119240 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:28:12.119240 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:28:12.119240 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:28:12.119240 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:28:12.120638 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:28:12.120638 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:28:12.120638 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 17:28:12.120638 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 17:28:12.120638 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 17:28:12.120638 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 17:28:12.120638 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 17:28:12.120638 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 17:28:12.120638 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 17:28:12.120638 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Aug 24 17:28:12.120638 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 358/704] 10.222.0.45 () {56 vars in 1093 bytes} [Wed Aug 24 17:28:11 2022] GET /identity/v3/users/nova => generated 82 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:28:12.121907 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d0ea2a-3fd0-49e1-a1a0-0dd596f004a8 None None] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:12.122190 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d0ea2a-3fd0-49e1-a1a0-0dd596f004a8 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:12.122511 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d0ea2a-3fd0-49e1-a1a0-0dd596f004a8 None None] PATH_INFO: `/` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:12.124210 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 347/705] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:28:12 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:28:12.124919 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef224c7-be6e-4e42-8cdb-dcc9d10930ec None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:12.129619 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0327ad37-654f-4f0e-a2df-214ac6ac064a None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:12.129619 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0327ad37-654f-4f0e-a2df-214ac6ac064a None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:12.129843 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0327ad37-654f-4f0e-a2df-214ac6ac064a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:12.158290 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-0327ad37-654f-4f0e-a2df-214ac6ac064a None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:28:12.159608 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-0327ad37-654f-4f0e-a2df-214ac6ac064a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:28:12.236392 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef224c7-be6e-4e42-8cdb-dcc9d10930ec None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:12.246274 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:12.272536 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d867b280-4a68-47bb-94cb-d47698431ef9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:12.273494 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d867b280-4a68-47bb-94cb-d47698431ef9 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:12.273997 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d867b280-4a68-47bb-94cb-d47698431ef9 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:12.274473 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d867b280-4a68-47bb-94cb-d47698431ef9 None admin] PATH_INFO: `/v3/users` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:12.275462 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d867b280-4a68-47bb-94cb-d47698431ef9 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:12.276715 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d867b280-4a68-47bb-94cb-d47698431ef9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:12.278463 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d867b280-4a68-47bb-94cb-d47698431ef9 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:12.283324 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 348/706] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:28:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 17:28:12.296974 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 359/707] 10.222.0.45 () {56 vars in 1133 bytes} [Wed Aug 24 17:28:12 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:12.305785 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0327ad37-654f-4f0e-a2df-214ac6ac064a None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:12.446252 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:12.456973 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0327ad37-654f-4f0e-a2df-214ac6ac064a None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:12.503891 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcde2bd-0c7b-4fe3-88fe-153dba8ce672 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:12.504502 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcde2bd-0c7b-4fe3-88fe-153dba8ce672 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:12.504800 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcde2bd-0c7b-4fe3-88fe-153dba8ce672 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:12.505068 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcde2bd-0c7b-4fe3-88fe-153dba8ce672 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:12.535667 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d867b280-4a68-47bb-94cb-d47698431ef9 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:12.604621 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfcde2bd-0c7b-4fe3-88fe-153dba8ce672 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:12.606507 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfcde2bd-0c7b-4fe3-88fe-153dba8ce672 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:12.668887 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d867b280-4a68-47bb-94cb-d47698431ef9 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:12.708136 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:12.712883 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa1a454-ca82-4375-bff5-29c8fadb52b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:12.713712 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa1a454-ca82-4375-bff5-29c8fadb52b7 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:12.713995 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa1a454-ca82-4375-bff5-29c8fadb52b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:12.714265 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa1a454-ca82-4375-bff5-29c8fadb52b7 None admin] PATH_INFO: `/v3/services/network` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:12.715191 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa1a454-ca82-4375-bff5-29c8fadb52b7 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:12.716889 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa1a454-ca82-4375-bff5-29c8fadb52b7 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:12.726533 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 349/708] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Aug 24 17:28:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 17:28:12.727794 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcde2bd-0c7b-4fe3-88fe-153dba8ce672 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:12.728913 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application [None req-cfa1a454-ca82-4375-bff5-29c8fadb52b7 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 17:28:12.728913 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:28:12.728913 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 17:28:12.728913 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 17:28:12.728913 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 17:28:12.728913 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 17:28:12.728913 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 17:28:12.728913 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:28:12.728913 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 17:28:12.728913 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Aug 24 17:28:12.728913 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 17:28:12.728913 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Aug 24 17:28:12.728913 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:28:12.728913 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:28:12.728913 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:28:12.728913 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:28:12.728913 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:28:12.730408 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:28:12.730408 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:28:12.730408 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:28:12.730408 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:28:12.730408 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:28:12.730408 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:28:12.730408 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 17:28:12.730408 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 17:28:12.730408 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 17:28:12.730408 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 17:28:12.730408 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:28:12.730408 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:28:12.730408 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:28:12.730408 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:28:12.730408 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:28:12.730408 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:28:12.731276 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:28:12.731276 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application with Lock( Aug 24 17:28:12.731276 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:28:12.731276 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:28:12.731276 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:28:12.731276 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:28:12.731276 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:28:12.731276 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:28:12.731276 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:28:12.731276 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:28:12.731276 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 17:28:12.731276 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:28:12.731276 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 17:28:12.731276 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Aug 24 17:28:12.731276 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 360/709] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Aug 24 17:28:12 2022] GET /identity/v3/services/network => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 17:28:12.732094 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa1a454-ca82-4375-bff5-29c8fadb52b7 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:12.771730 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:12.847974 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcde2bd-0c7b-4fe3-88fe-153dba8ce672 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:12.860032 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa1a454-ca82-4375-bff5-29c8fadb52b7 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:12.894149 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7d213c-661a-4519-9d2a-18fe1a223190 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:12.895092 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7d213c-661a-4519-9d2a-18fe1a223190 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:12.895382 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7d213c-661a-4519-9d2a-18fe1a223190 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:12.895658 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7d213c-661a-4519-9d2a-18fe1a223190 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:12.896519 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7d213c-661a-4519-9d2a-18fe1a223190 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:12.898075 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7d213c-661a-4519-9d2a-18fe1a223190 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:12.903567 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a106bf-4587-4b36-b80f-ea134d1ca610 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:12.904394 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a106bf-4587-4b36-b80f-ea134d1ca610 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:12.904884 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a106bf-4587-4b36-b80f-ea134d1ca610 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:12.905334 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a106bf-4587-4b36-b80f-ea134d1ca610 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:12.909555 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application [None req-9f7d213c-661a-4519-9d2a-18fe1a223190 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 17:28:12.909555 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:28:12.909555 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 17:28:12.909555 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 17:28:12.909555 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 17:28:12.909555 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 17:28:12.909555 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 17:28:12.909555 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:28:12.909555 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 17:28:12.909555 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Aug 24 17:28:12.909555 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 17:28:12.909555 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Aug 24 17:28:12.909555 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:28:12.909555 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:28:12.909555 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:28:12.909555 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:28:12.909555 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:28:12.910582 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:28:12.910582 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:28:12.910582 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:28:12.910582 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:28:12.910582 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:28:12.910582 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:28:12.910582 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 17:28:12.910582 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 17:28:12.910582 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 17:28:12.910582 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 17:28:12.910582 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:28:12.910582 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:28:12.910582 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:28:12.910582 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:28:12.910582 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:28:12.910582 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:28:12.911475 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:28:12.911475 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application with Lock( Aug 24 17:28:12.911475 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:28:12.911475 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:28:12.911475 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:28:12.911475 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:28:12.911475 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:28:12.911475 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:28:12.911475 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:28:12.911475 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:28:12.911475 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 17:28:12.911475 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:28:12.911475 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 17:28:12.911475 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Aug 24 17:28:12.911475 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 350/710] 10.222.0.45 () {56 vars in 1129 bytes} [Wed Aug 24 17:28:12 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:28:12.912299 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7d213c-661a-4519-9d2a-18fe1a223190 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:12.921021 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a106bf-4587-4b36-b80f-ea134d1ca610 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:12.922837 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a106bf-4587-4b36-b80f-ea134d1ca610 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:12.938723 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application [None req-93a106bf-4587-4b36-b80f-ea134d1ca610 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:28:12.938723 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:28:12.938723 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:28:12.938723 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:28:12.938723 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:28:12.938723 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:28:12.938723 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:28:12.938723 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:28:12.938723 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:28:12.938723 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:28:12.938723 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:28:12.938723 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:28:12.938723 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:28:12.938723 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:28:12.938723 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:28:12.938723 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:28:12.939666 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:28:12.939666 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:28:12.939666 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:28:12.939666 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:28:12.939666 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:28:12.939666 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:28:12.939666 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:28:12.939666 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application with Lock( Aug 24 17:28:12.939666 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:28:12.939666 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:28:12.939666 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:28:12.939666 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:28:12.939666 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:28:12.939666 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:28:12.939666 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:28:12.939666 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:28:12.939666 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:28:12.940589 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:28:12.940589 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:28:12.940589 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Aug 24 17:28:12.947030 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 361/711] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 17:28:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:28:12.948172 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a106bf-4587-4b36-b80f-ea134d1ca610 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:12.974959 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:13.031994 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7d213c-661a-4519-9d2a-18fe1a223190 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:13.038721 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:13.065048 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a106bf-4587-4b36-b80f-ea134d1ca610 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:13.073371 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ea3777-b09b-4992-8b49-c807a44e45cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:13.074240 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ea3777-b09b-4992-8b49-c807a44e45cf None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:13.074744 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ea3777-b09b-4992-8b49-c807a44e45cf None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:13.075292 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ea3777-b09b-4992-8b49-c807a44e45cf None admin] PATH_INFO: `/v3/services` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:13.076132 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ea3777-b09b-4992-8b49-c807a44e45cf None admin] RBAC: Adding query filter params (name=network) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:13.076716 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ea3777-b09b-4992-8b49-c807a44e45cf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:13.078436 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ea3777-b09b-4992-8b49-c807a44e45cf None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:13.084348 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 351/712] 10.222.0.45 () {56 vars in 1112 bytes} [Wed Aug 24 17:28:12 2022] GET /identity/v3/services?name=network => generated 124 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:13.085581 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ea3777-b09b-4992-8b49-c807a44e45cf None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:13.103444 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c86b89-d86f-4e80-a591-22acc2eb6cf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:13.104899 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c86b89-d86f-4e80-a591-22acc2eb6cf3 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:13.105439 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c86b89-d86f-4e80-a591-22acc2eb6cf3 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:13.105713 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c86b89-d86f-4e80-a591-22acc2eb6cf3 None admin] PATH_INFO: `/v3/services` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:13.106348 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c86b89-d86f-4e80-a591-22acc2eb6cf3 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:13.106652 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c86b89-d86f-4e80-a591-22acc2eb6cf3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:13.108472 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c86b89-d86f-4e80-a591-22acc2eb6cf3 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:13.114047 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 362/713] 10.222.0.45 () {56 vars in 1124 bytes} [Wed Aug 24 17:28:12 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:13.115022 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c86b89-d86f-4e80-a591-22acc2eb6cf3 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:13.214183 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ea3777-b09b-4992-8b49-c807a44e45cf None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:13.225722 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c86b89-d86f-4e80-a591-22acc2eb6cf3 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:13.236052 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:13.256908 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e11d2c5-f25d-4a2d-9ecd-ad8e63ad858e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:13.257524 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e11d2c5-f25d-4a2d-9ecd-ad8e63ad858e None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:13.257808 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e11d2c5-f25d-4a2d-9ecd-ad8e63ad858e None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:13.258393 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e11d2c5-f25d-4a2d-9ecd-ad8e63ad858e None admin] PATH_INFO: `/v3/domains` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:13.259318 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e11d2c5-f25d-4a2d-9ecd-ad8e63ad858e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:13.259627 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e11d2c5-f25d-4a2d-9ecd-ad8e63ad858e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:13.261219 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e11d2c5-f25d-4a2d-9ecd-ad8e63ad858e None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:13.266896 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd40bdf-1fa3-48b9-aa81-2c55bdb73980 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:13.267764 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd40bdf-1fa3-48b9-aa81-2c55bdb73980 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:13.268245 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd40bdf-1fa3-48b9-aa81-2c55bdb73980 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:13.268844 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd40bdf-1fa3-48b9-aa81-2c55bdb73980 None admin] PATH_INFO: `/v3/services` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:13.269696 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd40bdf-1fa3-48b9-aa81-2c55bdb73980 None admin] RBAC: Adding query filter params (type=network) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:13.270217 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd40bdf-1fa3-48b9-aa81-2c55bdb73980 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:13.271880 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd40bdf-1fa3-48b9-aa81-2c55bdb73980 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:13.278950 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 352/714] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 24 17:28:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:13.278950 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e11d2c5-f25d-4a2d-9ecd-ad8e63ad858e None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:13.283985 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 363/715] 10.222.0.45 () {56 vars in 1112 bytes} [Wed Aug 24 17:28:13 2022] GET /identity/v3/services?type=network => generated 124 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:13.285309 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd40bdf-1fa3-48b9-aa81-2c55bdb73980 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:13.302066 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:13.472891 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd40bdf-1fa3-48b9-aa81-2c55bdb73980 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:13.474723 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e11d2c5-f25d-4a2d-9ecd-ad8e63ad858e None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:13.516406 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b0c685-9958-4170-91a9-9cba4aea0396 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:13.517062 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b0c685-9958-4170-91a9-9cba4aea0396 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:13.517346 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b0c685-9958-4170-91a9-9cba4aea0396 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:13.517625 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b0c685-9958-4170-91a9-9cba4aea0396 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:13.524808 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b0c685-9958-4170-91a9-9cba4aea0396 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:13.526850 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b0c685-9958-4170-91a9-9cba4aea0396 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:13.538331 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application [None req-79b0c685-9958-4170-91a9-9cba4aea0396 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:28:13.538331 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:28:13.538331 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:28:13.538331 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:28:13.538331 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:28:13.538331 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:28:13.538331 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:28:13.538331 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:28:13.538331 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:28:13.538331 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:28:13.538331 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:28:13.538331 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:28:13.538331 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 17:28:13.538331 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 17:28:13.538331 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 17:28:13.539254 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 17:28:13.539254 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:28:13.539254 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:28:13.539254 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:28:13.539254 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:28:13.539254 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:28:13.539254 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:28:13.539254 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:28:13.539254 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application with Lock( Aug 24 17:28:13.539254 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:28:13.539254 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:28:13.539254 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:28:13.539254 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:28:13.539254 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:28:13.539254 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:28:13.539254 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:28:13.539254 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:28:13.540128 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 17:28:13.540128 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 17:28:13.540128 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 17:28:13.540128 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 17:28:13.540128 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 17:28:13.540128 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 17:28:13.540128 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:28:13.540128 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Aug 24 17:28:13.540128 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 364/716] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Aug 24 17:28:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:28:13.546765 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b0c685-9958-4170-91a9-9cba4aea0396 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:13.555230 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9b0481-c2f7-44c5-bef9-9f652721daeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:13.555871 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9b0481-c2f7-44c5-bef9-9f652721daeb None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:13.556151 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9b0481-c2f7-44c5-bef9-9f652721daeb None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:13.556423 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9b0481-c2f7-44c5-bef9-9f652721daeb None admin] PATH_INFO: `/v3/services` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:13.557126 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9b0481-c2f7-44c5-bef9-9f652721daeb None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:13.557432 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9b0481-c2f7-44c5-bef9-9f652721daeb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:13.558817 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9b0481-c2f7-44c5-bef9-9f652721daeb None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:13.566139 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 353/717] 10.222.0.45 () {56 vars in 1124 bytes} [Wed Aug 24 17:28:13 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:13.641583 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a126a15d-f6fb-4be5-aa61-ac465cc211d9 None None] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:13.641932 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a126a15d-f6fb-4be5-aa61-ac465cc211d9 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:13.642149 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a126a15d-f6fb-4be5-aa61-ac465cc211d9 None None] PATH_INFO: `/` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:13.644731 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 354/718] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:28:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:28:13.653633 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff217c1-5d75-47fb-b371-f6c73d0ef358 None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:13.653991 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff217c1-5d75-47fb-b371-f6c73d0ef358 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:13.654465 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff217c1-5d75-47fb-b371-f6c73d0ef358 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:13.683702 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:13.704395 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-3ff217c1-5d75-47fb-b371-f6c73d0ef358 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:28:13.705770 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-3ff217c1-5d75-47fb-b371-f6c73d0ef358 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:28:13.736150 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b0c685-9958-4170-91a9-9cba4aea0396 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:13.753258 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:13.780280 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c0c5e2-e01e-4197-8f4d-01aa489ae37d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:13.792579 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c0c5e2-e01e-4197-8f4d-01aa489ae37d None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:13.792579 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c0c5e2-e01e-4197-8f4d-01aa489ae37d None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:13.792579 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c0c5e2-e01e-4197-8f4d-01aa489ae37d None admin] PATH_INFO: `/v3/projects` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:13.792579 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c0c5e2-e01e-4197-8f4d-01aa489ae37d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:13.792579 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c0c5e2-e01e-4197-8f4d-01aa489ae37d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:13.792579 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c0c5e2-e01e-4197-8f4d-01aa489ae37d None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:13.800077 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 355/719] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:28:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:28:13.800464 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a7094c-84b4-4010-99fb-ec7668ceb27d None None] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:13.800904 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a7094c-84b4-4010-99fb-ec7668ceb27d None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:13.801078 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a7094c-84b4-4010-99fb-ec7668ceb27d None None] PATH_INFO: `/` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:13.804544 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 356/720] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:28:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:28:13.810081 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 365/721] 10.222.0.45 () {56 vars in 1148 bytes} [Wed Aug 24 17:28:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:28:13.811217 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b60fc8-f3ab-419b-9d0a-2e544aeefa44 None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:13.811406 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b60fc8-f3ab-419b-9d0a-2e544aeefa44 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:13.811681 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b60fc8-f3ab-419b-9d0a-2e544aeefa44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:13.816710 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c0c5e2-e01e-4197-8f4d-01aa489ae37d None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:13.848011 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-e5b60fc8-f3ab-419b-9d0a-2e544aeefa44 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:28:13.849230 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-e5b60fc8-f3ab-419b-9d0a-2e544aeefa44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:28:13.980676 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c0c5e2-e01e-4197-8f4d-01aa489ae37d None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:13.994228 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 357/722] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:28:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:28:14.032768 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:14.040252 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b3253a-b6fe-41bf-98ae-7de75d7d752f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:14.040930 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b3253a-b6fe-41bf-98ae-7de75d7d752f None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:14.041396 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b3253a-b6fe-41bf-98ae-7de75d7d752f None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:14.041671 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b3253a-b6fe-41bf-98ae-7de75d7d752f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:14.042401 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b3253a-b6fe-41bf-98ae-7de75d7d752f None admin] RBAC: Adding query filter params (role.id=e411ad9eb96e4e8d85b59b5b700ada7b, scope.project.id=2ad8a32bd9d2434097bd08b111914ca1, user.id=053b7ce6d6f649ed91a26e434dab805e) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:14.042706 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b3253a-b6fe-41bf-98ae-7de75d7d752f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:14.044129 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b3253a-b6fe-41bf-98ae-7de75d7d752f None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:14.052364 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 366/723] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Aug 24 17:28:13 2022] GET /identity/v3/role_assignments?user.id=053b7ce6d6f649ed91a26e434dab805e&scope.project.id=2ad8a32bd9d2434097bd08b111914ca1&role.id=e411ad9eb96e4e8d85b59b5b700ada7b => generated 259 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:28:14.096950 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b60fc8-f3ab-419b-9d0a-2e544aeefa44 None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:14.245888 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b60fc8-f3ab-419b-9d0a-2e544aeefa44 None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:14.247604 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:14.283746 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a2c73e-ed11-4454-8669-f9055aa67bd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:14.284304 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a2c73e-ed11-4454-8669-f9055aa67bd7 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:14.284602 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a2c73e-ed11-4454-8669-f9055aa67bd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:14.284877 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a2c73e-ed11-4454-8669-f9055aa67bd7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:14.297070 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a2c73e-ed11-4454-8669-f9055aa67bd7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:28:14.301922 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application [None req-82a2c73e-ed11-4454-8669-f9055aa67bd7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:28:14.301922 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:28:14.301922 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:28:14.301922 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:28:14.301922 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:28:14.301922 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:28:14.301922 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:28:14.301922 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:28:14.301922 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:28:14.301922 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:28:14.301922 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:28:14.301922 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:28:14.301922 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:28:14.301922 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:28:14.301922 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:28:14.301922 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:28:14.302882 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:28:14.302882 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:28:14.302882 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:28:14.302882 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:28:14.302882 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:28:14.302882 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:28:14.302882 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:28:14.302882 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:28:14.302882 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:28:14.302882 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:28:14.302882 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:28:14.302882 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application with Lock( Aug 24 17:28:14.302882 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:28:14.302882 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:28:14.302882 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:28:14.302882 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:28:14.303696 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:28:14.303696 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:28:14.303696 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:28:14.303696 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:28:14.303696 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:28:14.303696 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:28:14.303696 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:28:14.303696 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:28:14.303696 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:28:14.303696 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:28:14.303696 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:28:14.303696 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Aug 24 17:28:14.303696 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 358/724] 10.222.0.45 () {56 vars in 1096 bytes} [Wed Aug 24 17:28:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:28:14.313707 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b3253a-b6fe-41bf-98ae-7de75d7d752f None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:14.363366 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a2c73e-ed11-4454-8669-f9055aa67bd7 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:14.482970 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b3253a-b6fe-41bf-98ae-7de75d7d752f None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:14.527170 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a2c73e-ed11-4454-8669-f9055aa67bd7 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:14.551589 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cf3066-9f86-477a-b8f4-246f8d44bdb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:14.552133 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cf3066-9f86-477a-b8f4-246f8d44bdb8 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:14.552413 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cf3066-9f86-477a-b8f4-246f8d44bdb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:14.552711 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cf3066-9f86-477a-b8f4-246f8d44bdb8 None admin] PATH_INFO: `/v3/roles` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:14.553493 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cf3066-9f86-477a-b8f4-246f8d44bdb8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:14.553801 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cf3066-9f86-477a-b8f4-246f8d44bdb8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:14.555171 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cf3066-9f86-477a-b8f4-246f8d44bdb8 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:14.573246 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 367/725] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Aug 24 17:28:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:14.579645 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cf3066-9f86-477a-b8f4-246f8d44bdb8 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:14.603426 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6945a7a7-8ed7-40ff-8c96-535176708a02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:14.604617 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6945a7a7-8ed7-40ff-8c96-535176708a02 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:14.604897 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6945a7a7-8ed7-40ff-8c96-535176708a02 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:14.605419 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6945a7a7-8ed7-40ff-8c96-535176708a02 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:14.627606 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6945a7a7-8ed7-40ff-8c96-535176708a02 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:28:14.633395 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application [None req-6945a7a7-8ed7-40ff-8c96-535176708a02 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 17:28:14.633395 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:28:14.633395 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:28:14.633395 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:28:14.633395 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:28:14.633395 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:28:14.633395 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:28:14.633395 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:28:14.633395 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:28:14.633395 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:28:14.633395 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:28:14.633395 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:28:14.633395 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:28:14.633395 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:28:14.633395 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:28:14.633395 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:28:14.634301 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:28:14.634301 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:28:14.634301 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:28:14.634301 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:28:14.634301 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:28:14.634301 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:28:14.634301 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:28:14.634301 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:28:14.634301 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:28:14.634301 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:28:14.634301 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:28:14.634301 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application with Lock( Aug 24 17:28:14.634301 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:28:14.634301 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:28:14.634301 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:28:14.634301 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:28:14.635156 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:28:14.635156 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:28:14.635156 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:28:14.635156 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:28:14.635156 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:28:14.635156 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:28:14.635156 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:28:14.635156 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:28:14.635156 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:28:14.635156 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:28:14.635156 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 17:28:14.635156 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Aug 24 17:28:14.635156 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 359/726] 10.222.0.45 () {56 vars in 1102 bytes} [Wed Aug 24 17:28:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 17:28:14.641674 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6945a7a7-8ed7-40ff-8c96-535176708a02 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:14.681039 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:14.700853 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:14.755884 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6945a7a7-8ed7-40ff-8c96-535176708a02 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:14.793198 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97474bc8-609b-43cc-9cab-c75a3ebb1710 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:14.793963 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97474bc8-609b-43cc-9cab-c75a3ebb1710 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:14.794249 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97474bc8-609b-43cc-9cab-c75a3ebb1710 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:14.794525 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97474bc8-609b-43cc-9cab-c75a3ebb1710 None admin] PATH_INFO: `/v3/roles` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:14.795199 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97474bc8-609b-43cc-9cab-c75a3ebb1710 None admin] RBAC: Adding query filter params (name=service) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:14.795509 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97474bc8-609b-43cc-9cab-c75a3ebb1710 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:14.796921 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97474bc8-609b-43cc-9cab-c75a3ebb1710 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:14.806009 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 360/727] 10.222.0.45 () {56 vars in 1103 bytes} [Wed Aug 24 17:28:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:14.811969 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97474bc8-609b-43cc-9cab-c75a3ebb1710 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:14.862509 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cf3066-9f86-477a-b8f4-246f8d44bdb8 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:14.920924 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:14.925099 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97474bc8-609b-43cc-9cab-c75a3ebb1710 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:14.947218 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aecfba5-1eb0-440b-9dcc-5aeda73a7231 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:14.947891 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aecfba5-1eb0-440b-9dcc-5aeda73a7231 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:14.948169 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aecfba5-1eb0-440b-9dcc-5aeda73a7231 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:14.948440 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aecfba5-1eb0-440b-9dcc-5aeda73a7231 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:14.967407 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aecfba5-1eb0-440b-9dcc-5aeda73a7231 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:14.969436 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aecfba5-1eb0-440b-9dcc-5aeda73a7231 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:14.985193 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application [None req-4aecfba5-1eb0-440b-9dcc-5aeda73a7231 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:28:14.985193 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:28:14.985193 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:28:14.985193 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:28:14.985193 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:28:14.985193 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:28:14.985193 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:28:14.985193 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:28:14.985193 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:28:14.985193 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:28:14.985193 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:28:14.985193 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:28:14.985193 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:28:14.985193 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:28:14.985193 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:28:14.985193 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:28:14.986165 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:28:14.986165 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:28:14.986165 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:28:14.986165 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:28:14.986165 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:28:14.986165 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:28:14.986165 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:28:14.986165 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application with Lock( Aug 24 17:28:14.986165 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:28:14.986165 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:28:14.986165 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:28:14.986165 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:28:14.986165 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:28:14.986165 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:28:14.986165 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:28:14.986165 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:28:14.986165 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:28:14.987055 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:28:14.987055 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:28:14.987055 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Aug 24 17:28:14.987055 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 368/728] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 17:28:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 407 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:28:14.992309 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aecfba5-1eb0-440b-9dcc-5aeda73a7231 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:15.011116 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249f2874-f506-43c2-ad65-0ae4c0795ff8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:15.012073 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249f2874-f506-43c2-ad65-0ae4c0795ff8 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:15.012355 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249f2874-f506-43c2-ad65-0ae4c0795ff8 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:15.012646 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249f2874-f506-43c2-ad65-0ae4c0795ff8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:15.034585 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249f2874-f506-43c2-ad65-0ae4c0795ff8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:15.037038 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249f2874-f506-43c2-ad65-0ae4c0795ff8 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:15.069717 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application [None req-249f2874-f506-43c2-ad65-0ae4c0795ff8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:28:15.069717 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:28:15.069717 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:28:15.069717 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:28:15.069717 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:28:15.069717 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:28:15.069717 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:28:15.069717 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:28:15.069717 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:28:15.069717 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:28:15.069717 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:28:15.069717 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:28:15.069717 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:28:15.069717 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:28:15.069717 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:28:15.069717 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:28:15.070655 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:28:15.070655 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:28:15.070655 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:28:15.070655 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:28:15.070655 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:28:15.070655 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:28:15.070655 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:28:15.070655 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application with Lock( Aug 24 17:28:15.070655 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:28:15.070655 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:28:15.070655 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:28:15.070655 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:28:15.070655 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:28:15.070655 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:28:15.070655 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:28:15.070655 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:28:15.070655 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:28:15.071531 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:28:15.071531 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:28:15.071531 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Aug 24 17:28:15.071531 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 361/729] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 17:28:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 17:28:15.078224 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249f2874-f506-43c2-ad65-0ae4c0795ff8 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:15.141910 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aecfba5-1eb0-440b-9dcc-5aeda73a7231 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:15.151139 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:15.178697 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3c6faa-98c6-4720-94a9-8dc28a0f6776 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:15.179309 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3c6faa-98c6-4720-94a9-8dc28a0f6776 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:15.179587 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3c6faa-98c6-4720-94a9-8dc28a0f6776 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:15.179857 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3c6faa-98c6-4720-94a9-8dc28a0f6776 None admin] PATH_INFO: `/v3/domains` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:15.180497 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3c6faa-98c6-4720-94a9-8dc28a0f6776 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:15.180837 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3c6faa-98c6-4720-94a9-8dc28a0f6776 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:15.182763 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3c6faa-98c6-4720-94a9-8dc28a0f6776 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:15.191818 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249f2874-f506-43c2-ad65-0ae4c0795ff8 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:15.199251 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 369/730] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 24 17:28:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:15.200475 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3c6faa-98c6-4720-94a9-8dc28a0f6776 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:15.228478 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f36966-2b08-4b87-aa7d-176befd848c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:15.230294 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f36966-2b08-4b87-aa7d-176befd848c4 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:15.230783 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f36966-2b08-4b87-aa7d-176befd848c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:15.231386 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f36966-2b08-4b87-aa7d-176befd848c4 None admin] PATH_INFO: `/v3/domains` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:15.232283 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f36966-2b08-4b87-aa7d-176befd848c4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:15.232851 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f36966-2b08-4b87-aa7d-176befd848c4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:15.234425 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f36966-2b08-4b87-aa7d-176befd848c4 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:15.245412 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:15.249111 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 362/731] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 24 17:28:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:15.254154 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f36966-2b08-4b87-aa7d-176befd848c4 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:15.358930 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f36966-2b08-4b87-aa7d-176befd848c4 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:15.392710 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3c6faa-98c6-4720-94a9-8dc28a0f6776 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:15.403882 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68796a16-eae4-48f2-a48c-2baf80650421 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:15.404908 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68796a16-eae4-48f2-a48c-2baf80650421 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:15.405412 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68796a16-eae4-48f2-a48c-2baf80650421 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:15.405882 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68796a16-eae4-48f2-a48c-2baf80650421 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:15.419126 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68796a16-eae4-48f2-a48c-2baf80650421 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:15.422576 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68796a16-eae4-48f2-a48c-2baf80650421 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:15.451753 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application [None req-68796a16-eae4-48f2-a48c-2baf80650421 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 17:28:15.451753 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:28:15.451753 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:28:15.451753 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:28:15.451753 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:28:15.451753 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:28:15.451753 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:28:15.451753 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:28:15.451753 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:28:15.451753 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:28:15.451753 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:28:15.451753 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:28:15.451753 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 17:28:15.451753 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 17:28:15.451753 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 17:28:15.451753 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 17:28:15.452805 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:28:15.452805 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:28:15.452805 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 17:28:15.452805 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:28:15.452805 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 17:28:15.452805 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:28:15.452805 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:28:15.452805 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:28:15.452805 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:28:15.452805 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:28:15.452805 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:28:15.452805 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application with Lock( Aug 24 17:28:15.452805 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:28:15.452805 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:28:15.452805 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:28:15.452805 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:28:15.452805 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:28:15.452805 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:28:15.453754 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:28:15.453754 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:28:15.453754 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 17:28:15.453754 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 17:28:15.453754 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 17:28:15.453754 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 17:28:15.453754 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 17:28:15.453754 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 17:28:15.453754 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 17:28:15.453754 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Aug 24 17:28:15.458440 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 363/732] 10.222.0.45 () {56 vars in 1117 bytes} [Wed Aug 24 17:28:15 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:28:15.462498 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68796a16-eae4-48f2-a48c-2baf80650421 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:15.476873 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4137e95e-cdab-4c92-a772-92fdace8406f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:15.477919 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4137e95e-cdab-4c92-a772-92fdace8406f None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:15.478492 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4137e95e-cdab-4c92-a772-92fdace8406f None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:15.479074 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4137e95e-cdab-4c92-a772-92fdace8406f None admin] PATH_INFO: `/v3/users/swift` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:15.495233 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4137e95e-cdab-4c92-a772-92fdace8406f None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:15.497815 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4137e95e-cdab-4c92-a772-92fdace8406f None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:15.526594 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application [None req-4137e95e-cdab-4c92-a772-92fdace8406f None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 17:28:15.526594 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:28:15.526594 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:28:15.526594 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:28:15.526594 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:28:15.526594 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:28:15.526594 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:28:15.526594 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:28:15.526594 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:28:15.526594 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:28:15.526594 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:28:15.526594 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:28:15.526594 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 17:28:15.526594 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 17:28:15.526594 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 17:28:15.526594 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 17:28:15.527613 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:28:15.527613 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:28:15.527613 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 17:28:15.527613 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:28:15.527613 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 17:28:15.527613 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:28:15.527613 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:28:15.527613 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:28:15.527613 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:28:15.527613 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:28:15.527613 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:28:15.527613 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application with Lock( Aug 24 17:28:15.527613 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:28:15.527613 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:28:15.527613 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:28:15.527613 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:28:15.527613 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:28:15.527613 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:28:15.528601 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:28:15.528601 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:28:15.528601 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 17:28:15.528601 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 17:28:15.528601 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 17:28:15.528601 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 17:28:15.528601 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 17:28:15.528601 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 17:28:15.528601 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 17:28:15.528601 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Aug 24 17:28:15.534020 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 370/733] 10.222.0.45 () {56 vars in 1096 bytes} [Wed Aug 24 17:28:15 2022] GET /identity/v3/users/swift => generated 83 bytes in 334 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:28:15.539281 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4137e95e-cdab-4c92-a772-92fdace8406f None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:15.576573 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:15.586899 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:15.635644 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68796a16-eae4-48f2-a48c-2baf80650421 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:15.665911 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4137e95e-cdab-4c92-a772-92fdace8406f None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:15.678433 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689e494b-e0e0-460d-8133-ee52e5116d2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:15.679317 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689e494b-e0e0-460d-8133-ee52e5116d2d None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:15.679784 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689e494b-e0e0-460d-8133-ee52e5116d2d None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:15.680116 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689e494b-e0e0-460d-8133-ee52e5116d2d None admin] PATH_INFO: `/v3/users` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:15.680953 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689e494b-e0e0-460d-8133-ee52e5116d2d None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:15.681325 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689e494b-e0e0-460d-8133-ee52e5116d2d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:15.683606 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689e494b-e0e0-460d-8133-ee52e5116d2d None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:15.708925 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 364/734] 10.222.0.45 () {56 vars in 1149 bytes} [Wed Aug 24 17:28:15 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:15.709549 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4ab6a5-f96a-4d07-8ecc-a535a9099505 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:15.710233 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4ab6a5-f96a-4d07-8ecc-a535a9099505 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:15.710874 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4ab6a5-f96a-4d07-8ecc-a535a9099505 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:15.711180 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286160f1-7301-4824-b356-dd40d1092c37 None None] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:15.711457 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4ab6a5-f96a-4d07-8ecc-a535a9099505 None admin] PATH_INFO: `/v3/users` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:15.711823 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286160f1-7301-4824-b356-dd40d1092c37 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:15.712283 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286160f1-7301-4824-b356-dd40d1092c37 None None] PATH_INFO: `/` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:15.712837 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4ab6a5-f96a-4d07-8ecc-a535a9099505 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:15.713751 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4ab6a5-f96a-4d07-8ecc-a535a9099505 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:15.716603 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4ab6a5-f96a-4d07-8ecc-a535a9099505 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:15.718960 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 365/735] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:28:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:28:15.719878 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286160f1-7301-4824-b356-dd40d1092c37 None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:15.739860 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 371/736] 10.222.0.45 () {56 vars in 1135 bytes} [Wed Aug 24 17:28:15 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:15.741550 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b93ec1-a36b-4795-a959-02791f252c5e None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:15.741839 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b93ec1-a36b-4795-a959-02791f252c5e None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:15.742118 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b93ec1-a36b-4795-a959-02791f252c5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:15.766160 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-a6b93ec1-a36b-4795-a959-02791f252c5e None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:28:15.767822 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-a6b93ec1-a36b-4795-a959-02791f252c5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:28:15.832904 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286160f1-7301-4824-b356-dd40d1092c37 None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:15.848746 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:15.861169 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 372/737] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:28:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:28:15.861169 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b93ec1-a36b-4795-a959-02791f252c5e None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:15.876935 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93bfd52-b156-4792-a6e2-e9861c9c972c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:15.877614 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93bfd52-b156-4792-a6e2-e9861c9c972c None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:15.877895 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93bfd52-b156-4792-a6e2-e9861c9c972c None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:15.878169 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93bfd52-b156-4792-a6e2-e9861c9c972c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:15.885615 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:15.955787 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93bfd52-b156-4792-a6e2-e9861c9c972c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:15.957912 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93bfd52-b156-4792-a6e2-e9861c9c972c None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:16.001771 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b93ec1-a36b-4795-a959-02791f252c5e None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:16.052250 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d89a02-bc5d-4b43-8373-f92139a19b26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:16.053269 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d89a02-bc5d-4b43-8373-f92139a19b26 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:16.053646 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d89a02-bc5d-4b43-8373-f92139a19b26 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:16.054406 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d89a02-bc5d-4b43-8373-f92139a19b26 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:16.069424 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 366/738] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Aug 24 17:28:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 17:28:16.071031 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6375369-1ed1-45b1-9f3b-53bbc1f804cf None None] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:16.071515 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6375369-1ed1-45b1-9f3b-53bbc1f804cf None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:16.071970 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6375369-1ed1-45b1-9f3b-53bbc1f804cf None None] PATH_INFO: `/` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:16.073746 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 367/739] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:28:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:28:16.074635 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6375369-1ed1-45b1-9f3b-53bbc1f804cf None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:16.097588 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:16.137825 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d89a02-bc5d-4b43-8373-f92139a19b26 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:16.139666 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d89a02-bc5d-4b43-8373-f92139a19b26 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:16.159168 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:16.210976 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6375369-1ed1-45b1-9f3b-53bbc1f804cf None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:16.247159 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b296067-6705-4930-b3d7-d40137c9c98a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:16.252217 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 373/740] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Aug 24 17:28:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 17:28:16.252374 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b296067-6705-4930-b3d7-d40137c9c98a None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:16.252374 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b296067-6705-4930-b3d7-d40137c9c98a None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:16.252374 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b296067-6705-4930-b3d7-d40137c9c98a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:16.252627 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5cc5cf-6201-4c13-93d6-11db5789a8db None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:16.252627 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5cc5cf-6201-4c13-93d6-11db5789a8db None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:16.252627 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5cc5cf-6201-4c13-93d6-11db5789a8db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:16.263538 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b296067-6705-4930-b3d7-d40137c9c98a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:16.265034 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b296067-6705-4930-b3d7-d40137c9c98a None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:16.268308 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-6b5cc5cf-6201-4c13-93d6-11db5789a8db None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:28:16.269515 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-6b5cc5cf-6201-4c13-93d6-11db5789a8db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:28:16.280882 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application [None req-6b296067-6705-4930-b3d7-d40137c9c98a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:28:16.280882 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:28:16.280882 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:28:16.280882 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:28:16.280882 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:28:16.280882 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:28:16.280882 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:28:16.280882 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:28:16.280882 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:28:16.280882 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:28:16.280882 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:28:16.280882 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:28:16.280882 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:28:16.280882 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:28:16.280882 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:28:16.280882 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:28:16.281989 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:28:16.281989 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:28:16.281989 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:28:16.281989 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:28:16.281989 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:28:16.281989 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:28:16.281989 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:28:16.281989 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application with Lock( Aug 24 17:28:16.281989 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:28:16.281989 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:28:16.281989 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:28:16.281989 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:28:16.281989 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:28:16.281989 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:28:16.281989 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:28:16.281989 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:28:16.281989 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:28:16.282873 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:28:16.282873 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:28:16.282873 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Aug 24 17:28:16.282873 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 368/741] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 17:28:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:28:16.283100 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b296067-6705-4930-b3d7-d40137c9c98a None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:16.344688 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 374/742] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:28:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:28:16.345836 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5cc5cf-6201-4c13-93d6-11db5789a8db None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:16.352519 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:16.390977 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b296067-6705-4930-b3d7-d40137c9c98a None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:16.396191 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:16.426273 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a2f817-9273-48a0-b429-aa58b892fdc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:16.426880 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a2f817-9273-48a0-b429-aa58b892fdc2 None admin] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:16.427158 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a2f817-9273-48a0-b429-aa58b892fdc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:16.427432 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a2f817-9273-48a0-b429-aa58b892fdc2 None admin] PATH_INFO: `/v3/services` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:16.428073 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a2f817-9273-48a0-b429-aa58b892fdc2 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:16.429481 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a2f817-9273-48a0-b429-aa58b892fdc2 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:16.449694 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 369/743] 10.222.0.45 () {60 vars in 1141 bytes} [Wed Aug 24 17:28:16 2022] POST /identity/v3/services => generated 240 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:28:16.450937 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a2f817-9273-48a0-b429-aa58b892fdc2 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:16.456634 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5cc5cf-6201-4c13-93d6-11db5789a8db None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:16.519104 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9dff30d-d811-4b6d-bd60-2c40a555e96e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:16.520343 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dff30d-d811-4b6d-bd60-2c40a555e96e None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:16.520926 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dff30d-d811-4b6d-bd60-2c40a555e96e None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:16.521405 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dff30d-d811-4b6d-bd60-2c40a555e96e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:16.535917 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9dff30d-d811-4b6d-bd60-2c40a555e96e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:16.537970 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9dff30d-d811-4b6d-bd60-2c40a555e96e None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:16.554980 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application [None req-d9dff30d-d811-4b6d-bd60-2c40a555e96e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:28:16.554980 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:28:16.554980 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:28:16.554980 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:28:16.554980 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:28:16.554980 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:28:16.554980 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:28:16.554980 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:28:16.554980 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:28:16.554980 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:28:16.554980 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:28:16.554980 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:28:16.554980 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:28:16.554980 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:28:16.554980 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:28:16.554980 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:28:16.555984 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:28:16.555984 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:28:16.555984 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:28:16.555984 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:28:16.555984 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:28:16.555984 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:28:16.555984 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:28:16.555984 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application with Lock( Aug 24 17:28:16.555984 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:28:16.555984 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:28:16.555984 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:28:16.555984 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:28:16.555984 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:28:16.555984 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:28:16.555984 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:28:16.555984 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:28:16.555984 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:28:16.556909 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:28:16.556909 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:28:16.556909 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Aug 24 17:28:16.559328 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 375/744] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 17:28:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:28:16.560849 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d5e381-62b8-4dbd-8e87-24a4e049d1cf None None] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:16.561364 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d5e381-62b8-4dbd-8e87-24a4e049d1cf None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:16.562008 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d5e381-62b8-4dbd-8e87-24a4e049d1cf None None] PATH_INFO: `/` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:16.563868 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 376/745] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:28:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:28:16.564819 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d5e381-62b8-4dbd-8e87-24a4e049d1cf None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:16.573533 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a2f817-9273-48a0-b429-aa58b892fdc2 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:16.603268 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:16.619388 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871a9907-f98f-43b3-a0e8-63f319559200 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:16.619687 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871a9907-f98f-43b3-a0e8-63f319559200 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:16.620043 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871a9907-f98f-43b3-a0e8-63f319559200 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:16.620340 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871a9907-f98f-43b3-a0e8-63f319559200 None admin] PATH_INFO: `/v3/domains` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:16.621328 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871a9907-f98f-43b3-a0e8-63f319559200 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:16.621635 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871a9907-f98f-43b3-a0e8-63f319559200 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:16.624214 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871a9907-f98f-43b3-a0e8-63f319559200 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:16.637969 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 370/746] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 24 17:28:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:16.639498 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a7329f-19ed-40fd-bf38-4accaf190354 None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:16.639841 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a7329f-19ed-40fd-bf38-4accaf190354 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:16.640175 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a7329f-19ed-40fd-bf38-4accaf190354 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:16.658691 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-e3a7329f-19ed-40fd-bf38-4accaf190354 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:28:16.659888 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-e3a7329f-19ed-40fd-bf38-4accaf190354 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:28:16.679359 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:16.687597 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d5e381-62b8-4dbd-8e87-24a4e049d1cf None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:16.723000 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a68aad-3621-4a7c-910c-5c66dd7730cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:16.723612 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a68aad-3621-4a7c-910c-5c66dd7730cc None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:16.723895 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a68aad-3621-4a7c-910c-5c66dd7730cc None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:16.724173 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a68aad-3621-4a7c-910c-5c66dd7730cc None admin] PATH_INFO: `/v3/domains` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:16.724858 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a68aad-3621-4a7c-910c-5c66dd7730cc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:16.725156 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a68aad-3621-4a7c-910c-5c66dd7730cc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:16.726536 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a68aad-3621-4a7c-910c-5c66dd7730cc None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:16.732655 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 371/747] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:28:16 2022] POST /identity/v3/auth/tokens => generated 898 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:28:16.733331 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a7329f-19ed-40fd-bf38-4accaf190354 None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:16.747097 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 377/748] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 24 17:28:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:16.748207 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a68aad-3621-4a7c-910c-5c66dd7730cc None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:16.865073 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a7329f-19ed-40fd-bf38-4accaf190354 None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:16.882611 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:16.886623 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a68aad-3621-4a7c-910c-5c66dd7730cc None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:16.903144 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607d0bbd-0b1e-4177-b787-54a22184584a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:16.903755 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607d0bbd-0b1e-4177-b787-54a22184584a None admin] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:16.904033 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607d0bbd-0b1e-4177-b787-54a22184584a None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:16.904299 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607d0bbd-0b1e-4177-b787-54a22184584a None admin] PATH_INFO: `/v3/services` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:16.905048 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607d0bbd-0b1e-4177-b787-54a22184584a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:16.906412 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607d0bbd-0b1e-4177-b787-54a22184584a None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:16.923012 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 372/749] 10.222.0.45 () {60 vars in 1141 bytes} [Wed Aug 24 17:28:16 2022] POST /identity/v3/services => generated 251 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:28:16.924242 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607d0bbd-0b1e-4177-b787-54a22184584a None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:16.938837 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0bda92-3e55-437d-a919-6009709a3147 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:16.939833 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0bda92-3e55-437d-a919-6009709a3147 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:16.940291 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0bda92-3e55-437d-a919-6009709a3147 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:16.940774 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0bda92-3e55-437d-a919-6009709a3147 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:16.949112 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0bda92-3e55-437d-a919-6009709a3147 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:16.951052 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0bda92-3e55-437d-a919-6009709a3147 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:16.964166 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application [None req-5c0bda92-3e55-437d-a919-6009709a3147 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:28:16.964166 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:28:16.964166 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:28:16.964166 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:28:16.964166 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:28:16.964166 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:28:16.964166 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:28:16.964166 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:28:16.964166 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:28:16.964166 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:28:16.964166 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:28:16.964166 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:28:16.964166 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 17:28:16.964166 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 17:28:16.964166 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 17:28:16.965162 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 17:28:16.965162 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:28:16.965162 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:28:16.965162 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:28:16.965162 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:28:16.965162 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:28:16.965162 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:28:16.965162 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:28:16.965162 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application with Lock( Aug 24 17:28:16.965162 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:28:16.965162 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:28:16.965162 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:28:16.965162 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:28:16.965162 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:28:16.965162 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:28:16.965162 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:28:16.965162 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:28:16.966022 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 17:28:16.966022 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 17:28:16.966022 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 17:28:16.966022 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 17:28:16.966022 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 17:28:16.966022 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 17:28:16.966022 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:28:16.966022 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Aug 24 17:28:16.968504 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 378/750] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Aug 24 17:28:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:28:16.974670 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0bda92-3e55-437d-a919-6009709a3147 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:16.996574 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:17.041866 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607d0bbd-0b1e-4177-b787-54a22184584a None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:17.080230 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285bec95-5951-4d89-8424-cddd6f1d35fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:17.080928 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285bec95-5951-4d89-8424-cddd6f1d35fb None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:17.081204 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285bec95-5951-4d89-8424-cddd6f1d35fb None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:17.081471 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285bec95-5951-4d89-8424-cddd6f1d35fb None admin] PATH_INFO: `/v3/projects/service` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:17.089501 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285bec95-5951-4d89-8424-cddd6f1d35fb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:17.091203 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285bec95-5951-4d89-8424-cddd6f1d35fb None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:17.104965 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application [None req-285bec95-5951-4d89-8424-cddd6f1d35fb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:28:17.104965 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:28:17.104965 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:28:17.104965 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:28:17.104965 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:28:17.104965 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:28:17.104965 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:28:17.104965 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:28:17.104965 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:28:17.104965 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:28:17.104965 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:28:17.104965 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:28:17.104965 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 17:28:17.104965 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 17:28:17.104965 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 17:28:17.106167 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 17:28:17.106167 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:28:17.106167 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:28:17.106167 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:28:17.106167 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:28:17.106167 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:28:17.106167 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:28:17.106167 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:28:17.106167 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application with Lock( Aug 24 17:28:17.106167 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:28:17.106167 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:28:17.106167 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:28:17.106167 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:28:17.106167 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:28:17.106167 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:28:17.106167 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:28:17.106167 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:28:17.107039 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 17:28:17.107039 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 17:28:17.107039 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 17:28:17.107039 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 17:28:17.107039 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 17:28:17.107039 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 17:28:17.107039 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:28:17.107039 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Aug 24 17:28:17.107039 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 373/751] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Aug 24 17:28:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:28:17.112399 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285bec95-5951-4d89-8424-cddd6f1d35fb None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:17.140394 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0bda92-3e55-437d-a919-6009709a3147 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:17.159290 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:17.180476 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b7fb27-8651-4a31-8552-a0fccacdf6f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:17.181135 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b7fb27-8651-4a31-8552-a0fccacdf6f6 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:17.181396 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b7fb27-8651-4a31-8552-a0fccacdf6f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:17.181664 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b7fb27-8651-4a31-8552-a0fccacdf6f6 None admin] PATH_INFO: `/v3/projects` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:17.182891 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b7fb27-8651-4a31-8552-a0fccacdf6f6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:17.183312 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b7fb27-8651-4a31-8552-a0fccacdf6f6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:17.184769 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b7fb27-8651-4a31-8552-a0fccacdf6f6 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:17.199179 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 379/752] 10.222.0.45 () {56 vars in 1148 bytes} [Wed Aug 24 17:28:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:17.200133 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b7fb27-8651-4a31-8552-a0fccacdf6f6 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:17.241706 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285bec95-5951-4d89-8424-cddd6f1d35fb None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:17.279084 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b3be29-d772-463b-b181-fc929910d9aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:17.280030 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b3be29-d772-463b-b181-fc929910d9aa None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:17.280492 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b3be29-d772-463b-b181-fc929910d9aa None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:17.280976 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b3be29-d772-463b-b181-fc929910d9aa None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:17.285685 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:17.293410 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b3be29-d772-463b-b181-fc929910d9aa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:28:17.300492 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application [None req-77b3be29-d772-463b-b181-fc929910d9aa None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:28:17.300492 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:28:17.300492 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:28:17.300492 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:28:17.300492 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:28:17.300492 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:28:17.300492 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:28:17.300492 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:28:17.300492 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:28:17.300492 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:28:17.300492 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:28:17.300492 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:28:17.300492 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:28:17.300492 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:28:17.300492 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:28:17.300492 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:28:17.301454 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:28:17.301454 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:28:17.301454 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:28:17.301454 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:28:17.301454 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:28:17.301454 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:28:17.301454 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:28:17.301454 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:28:17.301454 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:28:17.301454 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:28:17.301454 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:28:17.301454 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application with Lock( Aug 24 17:28:17.301454 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:28:17.301454 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:28:17.301454 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:28:17.301454 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:28:17.302276 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:28:17.302276 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:28:17.302276 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:28:17.302276 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:28:17.302276 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:28:17.302276 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:28:17.302276 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:28:17.302276 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:28:17.302276 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:28:17.302276 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:28:17.302276 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:28:17.302276 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Aug 24 17:28:17.305613 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 374/753] 10.222.0.45 () {56 vars in 1096 bytes} [Wed Aug 24 17:28:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:28:17.311328 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b3be29-d772-463b-b181-fc929910d9aa None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:17.329508 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b7fb27-8651-4a31-8552-a0fccacdf6f6 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:17.367862 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a26ab3d-55e4-4b44-89fb-fab6f70b60e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:17.368951 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a26ab3d-55e4-4b44-89fb-fab6f70b60e0 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:17.370104 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a26ab3d-55e4-4b44-89fb-fab6f70b60e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:17.370593 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a26ab3d-55e4-4b44-89fb-fab6f70b60e0 None admin] PATH_INFO: `/v3/projects` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:17.371515 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a26ab3d-55e4-4b44-89fb-fab6f70b60e0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:17.372112 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a26ab3d-55e4-4b44-89fb-fab6f70b60e0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:17.373863 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a26ab3d-55e4-4b44-89fb-fab6f70b60e0 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:17.386991 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 380/754] 10.222.0.45 () {56 vars in 1148 bytes} [Wed Aug 24 17:28:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:17.388091 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a26ab3d-55e4-4b44-89fb-fab6f70b60e0 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:17.414033 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:17.421377 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b3be29-d772-463b-b181-fc929910d9aa None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:17.455675 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb52453-16ff-4f2e-a6d6-f55fd0a9fa37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:17.456328 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb52453-16ff-4f2e-a6d6-f55fd0a9fa37 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:17.456614 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb52453-16ff-4f2e-a6d6-f55fd0a9fa37 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:17.456925 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb52453-16ff-4f2e-a6d6-f55fd0a9fa37 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:17.457650 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb52453-16ff-4f2e-a6d6-f55fd0a9fa37 None admin] RBAC: Adding query filter params (role.id=ad6a81fb5d1941888577eb0ce9474882, scope.project.id=2ad8a32bd9d2434097bd08b111914ca1, user.id=ea143d042ee94ae7a4544d90d18db7db) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:17.457952 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb52453-16ff-4f2e-a6d6-f55fd0a9fa37 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:17.459590 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb52453-16ff-4f2e-a6d6-f55fd0a9fa37 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:17.465487 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 375/755] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Aug 24 17:28:17 2022] GET /identity/v3/role_assignments?user.id=ea143d042ee94ae7a4544d90d18db7db&scope.project.id=2ad8a32bd9d2434097bd08b111914ca1&role.id=ad6a81fb5d1941888577eb0ce9474882 => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:17.466614 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb52453-16ff-4f2e-a6d6-f55fd0a9fa37 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:17.544294 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a26ab3d-55e4-4b44-89fb-fab6f70b60e0 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:17.591767 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb52453-16ff-4f2e-a6d6-f55fd0a9fa37 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:17.610113 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:17.617501 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a37bda4-30db-4afc-82e9-ccc9e58fc232 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:17.618873 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a37bda4-30db-4afc-82e9-ccc9e58fc232 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:17.619164 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a37bda4-30db-4afc-82e9-ccc9e58fc232 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:17.620086 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a37bda4-30db-4afc-82e9-ccc9e58fc232 None admin] PATH_INFO: `/v3/roles` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:17.620907 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a37bda4-30db-4afc-82e9-ccc9e58fc232 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:17.621304 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a37bda4-30db-4afc-82e9-ccc9e58fc232 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:17.623439 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a37bda4-30db-4afc-82e9-ccc9e58fc232 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:17.635646 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 381/756] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Aug 24 17:28:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:17.638351 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423671ae-5cc5-48b8-bc3a-1a3456f4861a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:17.638966 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423671ae-5cc5-48b8-bc3a-1a3456f4861a None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:17.639240 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423671ae-5cc5-48b8-bc3a-1a3456f4861a None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:17.639508 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423671ae-5cc5-48b8-bc3a-1a3456f4861a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:17.640317 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423671ae-5cc5-48b8-bc3a-1a3456f4861a None admin] RBAC: Adding query filter params (role.id=e411ad9eb96e4e8d85b59b5b700ada7b, scope.project.id=2ad8a32bd9d2434097bd08b111914ca1, user.id=90f0772b64c84faaa689e7272fa97898) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:17.640665 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423671ae-5cc5-48b8-bc3a-1a3456f4861a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:17.641510 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a37bda4-30db-4afc-82e9-ccc9e58fc232 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:17.642083 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423671ae-5cc5-48b8-bc3a-1a3456f4861a None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:17.652694 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 376/757] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Aug 24 17:28:17 2022] GET /identity/v3/role_assignments?user.id=90f0772b64c84faaa689e7272fa97898&scope.project.id=2ad8a32bd9d2434097bd08b111914ca1&role.id=e411ad9eb96e4e8d85b59b5b700ada7b => generated 259 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:17.751590 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a37bda4-30db-4afc-82e9-ccc9e58fc232 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:17.790258 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de4edb9-563d-4df1-b8f8-615c0d0080a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:17.791252 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de4edb9-563d-4df1-b8f8-615c0d0080a0 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:17.792306 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de4edb9-563d-4df1-b8f8-615c0d0080a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:17.792827 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de4edb9-563d-4df1-b8f8-615c0d0080a0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:17.804119 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de4edb9-563d-4df1-b8f8-615c0d0080a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:17.805649 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de4edb9-563d-4df1-b8f8-615c0d0080a0 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:17.821001 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application [None req-9de4edb9-563d-4df1-b8f8-615c0d0080a0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:28:17.821001 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:28:17.821001 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:28:17.821001 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:28:17.821001 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:28:17.821001 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:28:17.821001 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:28:17.821001 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:28:17.821001 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:28:17.821001 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:28:17.821001 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:28:17.821001 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:28:17.821001 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:28:17.821001 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:28:17.821001 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:28:17.821001 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:28:17.822002 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:28:17.822002 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:28:17.822002 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:28:17.822002 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:28:17.822002 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:28:17.822002 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:28:17.822002 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:28:17.822002 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application with Lock( Aug 24 17:28:17.822002 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:28:17.822002 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:28:17.822002 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:28:17.822002 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:28:17.822002 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:28:17.822002 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:28:17.822002 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:28:17.822002 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:28:17.822002 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:28:17.822948 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:28:17.822948 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:28:17.822948 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Aug 24 17:28:17.822948 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 382/758] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 17:28:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:28:17.830140 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423671ae-5cc5-48b8-bc3a-1a3456f4861a None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:17.888684 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:17.969905 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423671ae-5cc5-48b8-bc3a-1a3456f4861a None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:18.017262 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d9aeae-0053-457e-89ff-e481a1c078d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:18.017882 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d9aeae-0053-457e-89ff-e481a1c078d3 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:18.018155 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d9aeae-0053-457e-89ff-e481a1c078d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:18.018422 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d9aeae-0053-457e-89ff-e481a1c078d3 None admin] PATH_INFO: `/v3/domains` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:18.019063 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d9aeae-0053-457e-89ff-e481a1c078d3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:18.019364 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d9aeae-0053-457e-89ff-e481a1c078d3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:18.020803 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d9aeae-0053-457e-89ff-e481a1c078d3 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:18.037061 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 377/759] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 24 17:28:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:18.042247 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de4edb9-563d-4df1-b8f8-615c0d0080a0 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:18.103899 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:18.167921 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de4edb9-563d-4df1-b8f8-615c0d0080a0 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:18.210656 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ba81ba-b15e-455e-af90-4c6a451965da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:18.211300 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ba81ba-b15e-455e-af90-4c6a451965da None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:18.211561 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ba81ba-b15e-455e-af90-4c6a451965da None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:18.211837 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ba81ba-b15e-455e-af90-4c6a451965da None admin] PATH_INFO: `/v3/users/nova` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:18.229672 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ba81ba-b15e-455e-af90-4c6a451965da None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:18.231413 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ba81ba-b15e-455e-af90-4c6a451965da None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:18.246218 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application [None req-50ba81ba-b15e-455e-af90-4c6a451965da None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 17:28:18.246218 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:28:18.246218 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:28:18.246218 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:28:18.246218 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:28:18.246218 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:28:18.246218 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:28:18.246218 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:28:18.246218 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:28:18.246218 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:28:18.246218 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:28:18.246218 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:28:18.246218 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 17:28:18.246218 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 17:28:18.246218 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 17:28:18.246218 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 17:28:18.247249 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:28:18.247249 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:28:18.247249 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 17:28:18.247249 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:28:18.247249 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 17:28:18.247249 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:28:18.247249 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:28:18.247249 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:28:18.247249 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:28:18.247249 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:28:18.247249 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:28:18.247249 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application with Lock( Aug 24 17:28:18.247249 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:28:18.247249 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:28:18.247249 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:28:18.247249 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:28:18.247249 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:28:18.247249 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:28:18.248171 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:28:18.248171 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:28:18.248171 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 17:28:18.248171 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 17:28:18.248171 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 17:28:18.248171 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 17:28:18.248171 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 17:28:18.248171 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 17:28:18.248171 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 17:28:18.248171 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Aug 24 17:28:18.248171 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 383/760] 10.222.0.45 () {56 vars in 1093 bytes} [Wed Aug 24 17:28:18 2022] GET /identity/v3/users/nova => generated 82 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:28:18.253299 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d9aeae-0053-457e-89ff-e481a1c078d3 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:18.383783 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:18.405206 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d9aeae-0053-457e-89ff-e481a1c078d3 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:18.440831 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfbfda2-0134-48fe-837a-d95e0a2a4dd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:18.444573 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfbfda2-0134-48fe-837a-d95e0a2a4dd7 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:18.444573 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfbfda2-0134-48fe-837a-d95e0a2a4dd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:18.444573 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfbfda2-0134-48fe-837a-d95e0a2a4dd7 None admin] PATH_INFO: `/v3/users` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:18.444573 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfbfda2-0134-48fe-837a-d95e0a2a4dd7 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:18.444573 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfbfda2-0134-48fe-837a-d95e0a2a4dd7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:18.444573 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfbfda2-0134-48fe-837a-d95e0a2a4dd7 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:18.464148 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 378/761] 10.222.0.45 () {56 vars in 1133 bytes} [Wed Aug 24 17:28:18 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:18.469371 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ba81ba-b15e-455e-af90-4c6a451965da None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:18.585683 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ba81ba-b15e-455e-af90-4c6a451965da None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:18.592883 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:18.621042 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ceb33d4-7d1d-441b-ae77-336429a13f5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:18.621680 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ceb33d4-7d1d-441b-ae77-336429a13f5d None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:18.621964 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ceb33d4-7d1d-441b-ae77-336429a13f5d None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:18.622239 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ceb33d4-7d1d-441b-ae77-336429a13f5d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:18.633084 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ceb33d4-7d1d-441b-ae77-336429a13f5d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:18.634558 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ceb33d4-7d1d-441b-ae77-336429a13f5d None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:18.649461 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application [None req-0ceb33d4-7d1d-441b-ae77-336429a13f5d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:28:18.649461 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:28:18.649461 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:28:18.649461 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:28:18.649461 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:28:18.649461 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:28:18.649461 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:28:18.649461 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:28:18.649461 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:28:18.649461 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:28:18.649461 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:28:18.649461 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:28:18.649461 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:28:18.649461 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:28:18.649461 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:28:18.649461 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:28:18.650429 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:28:18.650429 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:28:18.650429 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:28:18.650429 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:28:18.650429 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:28:18.650429 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:28:18.650429 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:28:18.650429 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application with Lock( Aug 24 17:28:18.650429 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:28:18.650429 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:28:18.650429 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:28:18.650429 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:28:18.650429 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:28:18.650429 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:28:18.650429 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:28:18.650429 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:28:18.650429 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:28:18.651352 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:28:18.651352 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:28:18.651352 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Aug 24 17:28:18.651352 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 384/762] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 17:28:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:28:18.656635 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfbfda2-0134-48fe-837a-d95e0a2a4dd7 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:18.792879 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfbfda2-0134-48fe-837a-d95e0a2a4dd7 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:18.828379 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e542b88d-3811-4a54-b499-7fbc654bd7df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:18.829019 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e542b88d-3811-4a54-b499-7fbc654bd7df None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:18.829300 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e542b88d-3811-4a54-b499-7fbc654bd7df None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:18.829573 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e542b88d-3811-4a54-b499-7fbc654bd7df None admin] PATH_INFO: `/v3/domains` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:18.830276 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e542b88d-3811-4a54-b499-7fbc654bd7df None admin] RBAC: Adding query filter params (name=Default) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:18.830583 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e542b88d-3811-4a54-b499-7fbc654bd7df None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:18.831944 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e542b88d-3811-4a54-b499-7fbc654bd7df None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:18.847748 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 379/763] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 24 17:28:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:18.853175 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ceb33d4-7d1d-441b-ae77-336429a13f5d None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:18.942343 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dae7c3d-a57a-428c-a867-4765f9bb9830 None None] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:18.942629 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dae7c3d-a57a-428c-a867-4765f9bb9830 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:18.942912 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dae7c3d-a57a-428c-a867-4765f9bb9830 None None] PATH_INFO: `/` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:18.944238 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 380/764] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:28:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 17:28:18.949428 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73db3559-a804-42f1-94a3-cd74a5bab53f None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:18.949839 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73db3559-a804-42f1-94a3-cd74a5bab53f None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:18.950215 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73db3559-a804-42f1-94a3-cd74a5bab53f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:18.956721 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:18.971094 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-73db3559-a804-42f1-94a3-cd74a5bab53f None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:28:18.972730 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-73db3559-a804-42f1-94a3-cd74a5bab53f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:28:18.996707 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ceb33d4-7d1d-441b-ae77-336429a13f5d None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:19.033191 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d89118b-89f1-46c7-87f5-2ebde78608a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:19.033826 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d89118b-89f1-46c7-87f5-2ebde78608a3 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:19.034109 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d89118b-89f1-46c7-87f5-2ebde78608a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:19.034383 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d89118b-89f1-46c7-87f5-2ebde78608a3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:19.041879 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d89118b-89f1-46c7-87f5-2ebde78608a3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:19.043384 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d89118b-89f1-46c7-87f5-2ebde78608a3 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:19.049135 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 381/765] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:28:18 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 17:28:19.054818 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application [None req-3d89118b-89f1-46c7-87f5-2ebde78608a3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:28:19.054818 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:28:19.054818 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:28:19.054818 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:28:19.054818 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:28:19.054818 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:28:19.054818 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:28:19.054818 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:28:19.054818 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:28:19.054818 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:28:19.054818 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:28:19.054818 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:28:19.054818 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 17:28:19.054818 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 17:28:19.054818 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 17:28:19.055756 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 17:28:19.055756 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:28:19.055756 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:28:19.055756 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:28:19.055756 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:28:19.055756 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:28:19.055756 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:28:19.055756 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:28:19.055756 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application with Lock( Aug 24 17:28:19.055756 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:28:19.055756 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:28:19.055756 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:28:19.055756 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:28:19.055756 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:28:19.055756 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:28:19.055756 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:28:19.055756 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:28:19.056664 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 17:28:19.056664 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 17:28:19.056664 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 17:28:19.056664 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 17:28:19.056664 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 17:28:19.056664 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 17:28:19.056664 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:28:19.056664 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Aug 24 17:28:19.056664 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 385/766] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Aug 24 17:28:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:28:19.062455 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73db3559-a804-42f1-94a3-cd74a5bab53f None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:19.112238 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754c26f3-22ce-4eb4-913d-3281f0ff94c4 None None] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:19.112608 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754c26f3-22ce-4eb4-913d-3281f0ff94c4 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:19.112832 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754c26f3-22ce-4eb4-913d-3281f0ff94c4 None None] PATH_INFO: `/` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:19.120572 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 386/767] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:28:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:28:19.120572 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747c7f44-90ac-44dc-ab45-80e8d2ac20bc None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:19.120572 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747c7f44-90ac-44dc-ab45-80e8d2ac20bc None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:19.120572 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747c7f44-90ac-44dc-ab45-80e8d2ac20bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:19.138474 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-747c7f44-90ac-44dc-ab45-80e8d2ac20bc None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:28:19.139727 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-747c7f44-90ac-44dc-ab45-80e8d2ac20bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:28:19.165037 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:19.195499 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73db3559-a804-42f1-94a3-cd74a5bab53f None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:19.225926 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:19.236574 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab00767-202d-4804-a658-f0332baaf302 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:19.236574 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab00767-202d-4804-a658-f0332baaf302 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:19.236574 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab00767-202d-4804-a658-f0332baaf302 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:19.236574 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab00767-202d-4804-a658-f0332baaf302 None admin] PATH_INFO: `/v3/projects` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:19.236945 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab00767-202d-4804-a658-f0332baaf302 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:19.236945 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab00767-202d-4804-a658-f0332baaf302 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:19.236945 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab00767-202d-4804-a658-f0332baaf302 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:19.248784 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 382/768] 10.222.0.45 () {56 vars in 1148 bytes} [Wed Aug 24 17:28:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:19.255017 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab00767-202d-4804-a658-f0332baaf302 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:19.286453 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 387/769] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:28:19 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 17:28:19.378133 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab00767-202d-4804-a658-f0332baaf302 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:19.412338 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76c6fcd-358a-4b8c-ac10-059a3bbf6081 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:19.413247 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76c6fcd-358a-4b8c-ac10-059a3bbf6081 None admin] REQUEST_METHOD: `PUT` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:19.413675 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76c6fcd-358a-4b8c-ac10-059a3bbf6081 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:19.414089 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76c6fcd-358a-4b8c-ac10-059a3bbf6081 None admin] PATH_INFO: `/v3/projects/2ad8a32bd9d2434097bd08b111914ca1/users/053b7ce6d6f649ed91a26e434dab805e/roles/e411ad9eb96e4e8d85b59b5b700ada7b` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:19.416160 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76c6fcd-358a-4b8c-ac10-059a3bbf6081 None admin] RBAC: Authorizing `identity:create_grant(project_id=2ad8a32bd9d2434097bd08b111914ca1, user_id=053b7ce6d6f649ed91a26e434dab805e, role_id=e411ad9eb96e4e8d85b59b5b700ada7b)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:19.418314 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76c6fcd-358a-4b8c-ac10-059a3bbf6081 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:19.437048 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747c7f44-90ac-44dc-ab45-80e8d2ac20bc None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:19.439099 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 383/770] 10.222.0.45 () {58 vars in 1439 bytes} [Wed Aug 24 17:28:19 2022] PUT /identity/v3/projects/2ad8a32bd9d2434097bd08b111914ca1/users/053b7ce6d6f649ed91a26e434dab805e/roles/e411ad9eb96e4e8d85b59b5b700ada7b => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:28:19.599834 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747c7f44-90ac-44dc-ab45-80e8d2ac20bc None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:19.641879 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76c6fcd-358a-4b8c-ac10-059a3bbf6081 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:19.651791 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f420fc1-4eb3-4be3-b326-f8e76c77faad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:19.652495 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f420fc1-4eb3-4be3-b326-f8e76c77faad None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:19.652810 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f420fc1-4eb3-4be3-b326-f8e76c77faad None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:19.653080 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f420fc1-4eb3-4be3-b326-f8e76c77faad None admin] PATH_INFO: `/v3/services/network` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:19.653733 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f420fc1-4eb3-4be3-b326-f8e76c77faad None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:19.655113 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f420fc1-4eb3-4be3-b326-f8e76c77faad None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:19.669760 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application [None req-7f420fc1-4eb3-4be3-b326-f8e76c77faad None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 17:28:19.669760 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:28:19.669760 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 17:28:19.669760 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 17:28:19.669760 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 17:28:19.669760 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 17:28:19.669760 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 17:28:19.669760 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:28:19.669760 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 17:28:19.669760 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Aug 24 17:28:19.669760 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 17:28:19.669760 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Aug 24 17:28:19.669760 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:28:19.669760 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:28:19.669760 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:28:19.669760 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:28:19.669760 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:28:19.670805 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:28:19.670805 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:28:19.670805 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:28:19.670805 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:28:19.670805 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:28:19.670805 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:28:19.670805 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 17:28:19.670805 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 17:28:19.670805 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 17:28:19.670805 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 17:28:19.670805 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:28:19.670805 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:28:19.670805 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:28:19.670805 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:28:19.670805 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:28:19.670805 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:28:19.672257 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:28:19.672257 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application with Lock( Aug 24 17:28:19.672257 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:28:19.672257 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:28:19.672257 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:28:19.672257 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:28:19.672257 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:28:19.672257 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:28:19.672257 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:28:19.672257 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:28:19.672257 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 17:28:19.672257 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:28:19.672257 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 17:28:19.672257 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Aug 24 17:28:19.672257 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 388/771] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Aug 24 17:28:19 2022] GET /identity/v3/services/network => generated 88 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:28:19.676765 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f420fc1-4eb3-4be3-b326-f8e76c77faad None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:19.701823 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:19.721194 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:19.786690 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76c6fcd-358a-4b8c-ac10-059a3bbf6081 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:19.815376 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f420fc1-4eb3-4be3-b326-f8e76c77faad None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:19.831030 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6090ed4-f078-4fc1-b7e7-b62e1febe09e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:19.832835 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6090ed4-f078-4fc1-b7e7-b62e1febe09e None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:19.833119 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6090ed4-f078-4fc1-b7e7-b62e1febe09e None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:19.833392 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6090ed4-f078-4fc1-b7e7-b62e1febe09e None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:19.833985 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6090ed4-f078-4fc1-b7e7-b62e1febe09e None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:19.835379 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6090ed4-f078-4fc1-b7e7-b62e1febe09e None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:19.849294 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application [None req-f6090ed4-f078-4fc1-b7e7-b62e1febe09e None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 17:28:19.849294 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:28:19.849294 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 17:28:19.849294 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 17:28:19.849294 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 17:28:19.849294 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 17:28:19.849294 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 17:28:19.849294 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:28:19.849294 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 17:28:19.849294 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Aug 24 17:28:19.849294 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 17:28:19.849294 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Aug 24 17:28:19.849294 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:28:19.849294 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:28:19.849294 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:28:19.849294 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:28:19.849294 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:28:19.850319 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:28:19.850319 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:28:19.850319 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:28:19.850319 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:28:19.850319 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:28:19.850319 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:28:19.850319 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 17:28:19.850319 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 17:28:19.850319 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 17:28:19.850319 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 17:28:19.850319 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:28:19.850319 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:28:19.850319 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:28:19.850319 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:28:19.850319 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:28:19.850319 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:28:19.851140 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:28:19.851140 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application with Lock( Aug 24 17:28:19.851140 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:28:19.851140 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:28:19.851140 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:28:19.851140 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:28:19.851140 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:28:19.851140 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:28:19.851140 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:28:19.851140 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:28:19.851140 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 17:28:19.851140 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:28:19.851140 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 17:28:19.851140 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Aug 24 17:28:19.851140 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 384/772] 10.222.0.45 () {56 vars in 1129 bytes} [Wed Aug 24 17:28:19 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 17:28:19.857700 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6090ed4-f078-4fc1-b7e7-b62e1febe09e None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:19.888526 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1eef5b2-3fd6-4067-b271-3c7439f43f8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:19.889170 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eef5b2-3fd6-4067-b271-3c7439f43f8b None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:19.889442 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eef5b2-3fd6-4067-b271-3c7439f43f8b None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:19.889708 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eef5b2-3fd6-4067-b271-3c7439f43f8b None admin] PATH_INFO: `/v3/services` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:19.890396 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1eef5b2-3fd6-4067-b271-3c7439f43f8b None admin] RBAC: Adding query filter params (name=network) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:19.890696 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1eef5b2-3fd6-4067-b271-3c7439f43f8b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:19.892121 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1eef5b2-3fd6-4067-b271-3c7439f43f8b None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:19.903402 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 389/773] 10.222.0.45 () {56 vars in 1112 bytes} [Wed Aug 24 17:28:19 2022] GET /identity/v3/services?name=network => generated 124 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:28:19.908801 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1eef5b2-3fd6-4067-b271-3c7439f43f8b None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:20.006402 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:20.007647 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6090ed4-f078-4fc1-b7e7-b62e1febe09e None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:20.032773 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:20.033667 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1eef5b2-3fd6-4067-b271-3c7439f43f8b None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:20.058717 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d72ed61-1fa2-4148-9995-9b4ff915e2f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:20.059952 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d72ed61-1fa2-4148-9995-9b4ff915e2f5 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:20.060732 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d72ed61-1fa2-4148-9995-9b4ff915e2f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:20.061626 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d72ed61-1fa2-4148-9995-9b4ff915e2f5 None admin] PATH_INFO: `/v3/services` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:20.062737 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d72ed61-1fa2-4148-9995-9b4ff915e2f5 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:20.063504 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d72ed61-1fa2-4148-9995-9b4ff915e2f5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:20.066902 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d72ed61-1fa2-4148-9995-9b4ff915e2f5 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:20.072376 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7edc470-7bdf-4200-bb1b-fe158c9ad748 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:20.073578 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7edc470-7bdf-4200-bb1b-fe158c9ad748 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:20.073880 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7edc470-7bdf-4200-bb1b-fe158c9ad748 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:20.074138 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7edc470-7bdf-4200-bb1b-fe158c9ad748 None admin] PATH_INFO: `/v3/services` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:20.074762 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7edc470-7bdf-4200-bb1b-fe158c9ad748 None admin] RBAC: Adding query filter params (type=network) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:20.075064 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7edc470-7bdf-4200-bb1b-fe158c9ad748 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:20.076446 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7edc470-7bdf-4200-bb1b-fe158c9ad748 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:20.079256 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 385/774] 10.222.0.45 () {56 vars in 1124 bytes} [Wed Aug 24 17:28:19 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:20.081139 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32582a0-52dd-4c89-b528-95ca0640a9ee None None] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:20.081632 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32582a0-52dd-4c89-b528-95ca0640a9ee None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:20.082065 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 390/775] 10.222.0.45 () {56 vars in 1112 bytes} [Wed Aug 24 17:28:19 2022] GET /identity/v3/services?type=network => generated 350 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:20.082327 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32582a0-52dd-4c89-b528-95ca0640a9ee None None] PATH_INFO: `/` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:20.084203 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 386/776] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:28:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:28:20.088426 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7edc470-7bdf-4200-bb1b-fe158c9ad748 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:20.089445 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32582a0-52dd-4c89-b528-95ca0640a9ee None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:20.199529 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7edc470-7bdf-4200-bb1b-fe158c9ad748 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:20.203683 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32582a0-52dd-4c89-b528-95ca0640a9ee None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:20.234637 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3936c0-f37d-4966-ab09-48be4f1afee0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:20.235240 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3936c0-f37d-4966-ab09-48be4f1afee0 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:20.235523 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3936c0-f37d-4966-ab09-48be4f1afee0 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:20.235802 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3936c0-f37d-4966-ab09-48be4f1afee0 None admin] PATH_INFO: `/v3/services` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:20.236430 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3936c0-f37d-4966-ab09-48be4f1afee0 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:20.236779 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3936c0-f37d-4966-ab09-48be4f1afee0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:20.238130 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3936c0-f37d-4966-ab09-48be4f1afee0 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:20.240970 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173b0763-2b2a-428c-9d72-64254dc76e35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:20.241578 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173b0763-2b2a-428c-9d72-64254dc76e35 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:20.241855 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173b0763-2b2a-428c-9d72-64254dc76e35 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:20.242127 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173b0763-2b2a-428c-9d72-64254dc76e35 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:20.242820 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173b0763-2b2a-428c-9d72-64254dc76e35 None admin] RBAC: Adding query filter params (interface=public, service_id=90ddab15c820434da9a4b336766a0560, region_id=RegionOne) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:20.248283 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 391/777] 10.222.0.45 () {56 vars in 1124 bytes} [Wed Aug 24 17:28:20 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:20.248423 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173b0763-2b2a-428c-9d72-64254dc76e35 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:20.248515 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2896a2d6-abd1-4c7f-8a31-df0910d4291b None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:20.248644 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173b0763-2b2a-428c-9d72-64254dc76e35 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:20.248744 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2896a2d6-abd1-4c7f-8a31-df0910d4291b None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:20.248744 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2896a2d6-abd1-4c7f-8a31-df0910d4291b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:20.251483 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:20.260785 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 387/778] 10.222.0.45 () {56 vars in 1251 bytes} [Wed Aug 24 17:28:20 2022] GET /identity/v3/endpoints?service_id=90ddab15c820434da9a4b336766a0560&interface=public®ion_id=RegionOne => generated 194 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:20.260785 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dd9470-8fe8-4fca-9883-72e4e34257c5 None None] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:20.260785 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dd9470-8fe8-4fca-9883-72e4e34257c5 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:20.260785 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dd9470-8fe8-4fca-9883-72e4e34257c5 None None] PATH_INFO: `/` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:20.260785 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 388/779] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:28:20 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:28:20.260785 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34dd9470-8fe8-4fca-9883-72e4e34257c5 None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:20.267603 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-2896a2d6-abd1-4c7f-8a31-df0910d4291b None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:28:20.268883 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-2896a2d6-abd1-4c7f-8a31-df0910d4291b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:28:20.309542 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:20.336229 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 392/780] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:28:20 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:28:20.337354 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2896a2d6-abd1-4c7f-8a31-df0910d4291b None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:20.378629 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34dd9470-8fe8-4fca-9883-72e4e34257c5 None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:20.414706 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa92b86-6455-46fb-b32c-a74f28b4d24d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:20.415648 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa92b86-6455-46fb-b32c-a74f28b4d24d None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:20.416122 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa92b86-6455-46fb-b32c-a74f28b4d24d None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:20.417182 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa92b86-6455-46fb-b32c-a74f28b4d24d None admin] PATH_INFO: `/v3/endpoints` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:20.418072 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa92b86-6455-46fb-b32c-a74f28b4d24d None admin] RBAC: Adding query filter params (interface=public, service_id=8fed738ef09d4e9bac7db763858d599e, region_id=RegionOne) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:20.418594 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa92b86-6455-46fb-b32c-a74f28b4d24d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:20.420302 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa92b86-6455-46fb-b32c-a74f28b4d24d None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:20.430509 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 389/781] 10.222.0.45 () {56 vars in 1251 bytes} [Wed Aug 24 17:28:20 2022] GET /identity/v3/endpoints?service_id=8fed738ef09d4e9bac7db763858d599e&interface=public®ion_id=RegionOne => generated 194 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:20.432050 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623e4493-afd2-42e3-9ac7-2e3ba87482fe None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:20.433149 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623e4493-afd2-42e3-9ac7-2e3ba87482fe None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:20.433755 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623e4493-afd2-42e3-9ac7-2e3ba87482fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:20.443212 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2896a2d6-abd1-4c7f-8a31-df0910d4291b None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:20.452798 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-623e4493-afd2-42e3-9ac7-2e3ba87482fe None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:28:20.454293 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-623e4493-afd2-42e3-9ac7-2e3ba87482fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:28:20.473603 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:20.480206 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbe6f1e-15e9-4905-a4c2-b56a7c2aad11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:20.481155 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbe6f1e-15e9-4905-a4c2-b56a7c2aad11 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:20.481418 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbe6f1e-15e9-4905-a4c2-b56a7c2aad11 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:20.481692 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbe6f1e-15e9-4905-a4c2-b56a7c2aad11 None admin] PATH_INFO: `/v3/services` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:20.482342 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbe6f1e-15e9-4905-a4c2-b56a7c2aad11 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:20.483896 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbe6f1e-15e9-4905-a4c2-b56a7c2aad11 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:20.496578 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 393/782] 10.222.0.45 () {56 vars in 1087 bytes} [Wed Aug 24 17:28:20 2022] GET /identity/v3/services => generated 772 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:20.496578 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbe6f1e-15e9-4905-a4c2-b56a7c2aad11 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:20.586550 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 390/783] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:28:20 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:28:20.642784 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623e4493-afd2-42e3-9ac7-2e3ba87482fe None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:20.652935 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbe6f1e-15e9-4905-a4c2-b56a7c2aad11 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:20.700991 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:20.703770 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5dbaafd-ecb6-4988-ab23-9f15ea58aa0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:20.705022 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dbaafd-ecb6-4988-ab23-9f15ea58aa0c None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:20.705306 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dbaafd-ecb6-4988-ab23-9f15ea58aa0c None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:20.705578 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dbaafd-ecb6-4988-ab23-9f15ea58aa0c None admin] PATH_INFO: `/v3/services` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:20.706186 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5dbaafd-ecb6-4988-ab23-9f15ea58aa0c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:20.708047 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5dbaafd-ecb6-4988-ab23-9f15ea58aa0c None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:20.720582 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 394/784] 10.222.0.45 () {56 vars in 1087 bytes} [Wed Aug 24 17:28:20 2022] GET /identity/v3/services => generated 772 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:20.778943 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623e4493-afd2-42e3-9ac7-2e3ba87482fe None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:20.820893 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ca7911-6960-4186-af69-69db37257223 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:20.821561 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ca7911-6960-4186-af69-69db37257223 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:20.821846 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ca7911-6960-4186-af69-69db37257223 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:20.822170 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ca7911-6960-4186-af69-69db37257223 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:20.839161 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ca7911-6960-4186-af69-69db37257223 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:28:20.849816 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application [None req-35ca7911-6960-4186-af69-69db37257223 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 17:28:20.849816 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:28:20.849816 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:28:20.849816 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:28:20.849816 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:28:20.849816 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:28:20.849816 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:28:20.849816 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:28:20.849816 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:28:20.849816 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:28:20.849816 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:28:20.849816 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:28:20.849816 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:28:20.849816 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:28:20.849816 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:28:20.849816 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:28:20.850773 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:28:20.850773 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:28:20.850773 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:28:20.850773 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:28:20.850773 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:28:20.850773 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:28:20.850773 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:28:20.850773 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:28:20.850773 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:28:20.850773 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:28:20.850773 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:28:20.850773 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application with Lock( Aug 24 17:28:20.850773 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:28:20.850773 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:28:20.850773 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:28:20.850773 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:28:20.851603 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:28:20.851603 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:28:20.851603 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:28:20.851603 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:28:20.851603 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:28:20.851603 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:28:20.851603 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:28:20.851603 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:28:20.851603 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:28:20.851603 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:28:20.851603 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 17:28:20.851603 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Aug 24 17:28:20.851603 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 391/785] 10.222.0.45 () {56 vars in 1102 bytes} [Wed Aug 24 17:28:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:28:20.859480 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5dbaafd-ecb6-4988-ab23-9f15ea58aa0c None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:20.918296 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ca7911-6960-4186-af69-69db37257223 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:20.951707 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:21.014876 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5dbaafd-ecb6-4988-ab23-9f15ea58aa0c None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:21.058963 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e98c66-cd74-49db-8fd6-0e96ae4bbd5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:21.058963 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e98c66-cd74-49db-8fd6-0e96ae4bbd5f None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:21.060053 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e98c66-cd74-49db-8fd6-0e96ae4bbd5f None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:21.060053 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e98c66-cd74-49db-8fd6-0e96ae4bbd5f None admin] PATH_INFO: `/v3/roles` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:21.060327 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e98c66-cd74-49db-8fd6-0e96ae4bbd5f None admin] RBAC: Adding query filter params (name=service) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:21.061232 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e98c66-cd74-49db-8fd6-0e96ae4bbd5f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:21.062698 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e98c66-cd74-49db-8fd6-0e96ae4bbd5f None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:21.067309 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:21.073761 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ca7911-6960-4186-af69-69db37257223 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:21.081685 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 395/786] 10.222.0.45 () {56 vars in 1103 bytes} [Wed Aug 24 17:28:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:21.083023 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e98c66-cd74-49db-8fd6-0e96ae4bbd5f None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:21.179206 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f186f3-6a50-46fa-b40e-021d9034807d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:21.181222 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f186f3-6a50-46fa-b40e-021d9034807d None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:21.182831 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f186f3-6a50-46fa-b40e-021d9034807d None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:21.193776 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f186f3-6a50-46fa-b40e-021d9034807d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:21.237246 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f186f3-6a50-46fa-b40e-021d9034807d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:28:21.243186 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application [None req-41f186f3-6a50-46fa-b40e-021d9034807d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:28:21.243186 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:28:21.243186 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:28:21.243186 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:28:21.243186 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:28:21.243186 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:28:21.243186 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:28:21.243186 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:28:21.243186 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:28:21.243186 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:28:21.243186 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:28:21.243186 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:28:21.243186 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:28:21.243186 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:28:21.243186 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:28:21.243186 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:28:21.244105 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:28:21.244105 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:28:21.244105 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:28:21.244105 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:28:21.244105 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:28:21.244105 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:28:21.244105 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:28:21.244105 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:28:21.244105 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:28:21.244105 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:28:21.244105 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:28:21.244105 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application with Lock( Aug 24 17:28:21.244105 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:28:21.244105 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:28:21.244105 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:28:21.244105 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:28:21.245031 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:28:21.245031 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:28:21.245031 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:28:21.245031 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:28:21.245031 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:28:21.245031 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:28:21.245031 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:28:21.245031 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:28:21.245031 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:28:21.245031 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:28:21.245031 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:28:21.245031 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Aug 24 17:28:21.263490 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 392/787] 10.222.0.45 () {56 vars in 1096 bytes} [Wed Aug 24 17:28:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 345 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 17:28:21.266225 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f186f3-6a50-46fa-b40e-021d9034807d None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:21.379173 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e98c66-cd74-49db-8fd6-0e96ae4bbd5f None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:21.462295 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:21.486258 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c263d37d-c7d1-4fdd-bea7-021d5b7779a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:21.486915 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c263d37d-c7d1-4fdd-bea7-021d5b7779a8 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:21.487197 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c263d37d-c7d1-4fdd-bea7-021d5b7779a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:21.487476 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c263d37d-c7d1-4fdd-bea7-021d5b7779a8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:21.525432 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c263d37d-c7d1-4fdd-bea7-021d5b7779a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:21.526849 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c263d37d-c7d1-4fdd-bea7-021d5b7779a8 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:21.565518 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application [None req-c263d37d-c7d1-4fdd-bea7-021d5b7779a8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:28:21.565518 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:28:21.565518 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:28:21.565518 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:28:21.565518 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:28:21.565518 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:28:21.565518 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:28:21.565518 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:28:21.565518 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:28:21.565518 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:28:21.565518 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:28:21.565518 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:28:21.565518 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:28:21.565518 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:28:21.565518 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:28:21.565518 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:28:21.566521 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:28:21.566521 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:28:21.566521 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:28:21.566521 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:28:21.566521 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:28:21.566521 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:28:21.566521 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:28:21.566521 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application with Lock( Aug 24 17:28:21.566521 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:28:21.566521 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:28:21.566521 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:28:21.566521 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:28:21.566521 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:28:21.566521 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:28:21.566521 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:28:21.566521 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:28:21.566521 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:28:21.567408 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:28:21.567408 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:28:21.567408 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Aug 24 17:28:21.567408 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 396/788] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 17:28:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 484 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 17:28:21.575074 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c263d37d-c7d1-4fdd-bea7-021d5b7779a8 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:21.582470 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f186f3-6a50-46fa-b40e-021d9034807d None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:21.634824 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464d47d5-334e-4713-9ce7-cea9575f2b76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:21.636372 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464d47d5-334e-4713-9ce7-cea9575f2b76 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:21.636870 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464d47d5-334e-4713-9ce7-cea9575f2b76 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:21.637341 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464d47d5-334e-4713-9ce7-cea9575f2b76 None admin] PATH_INFO: `/v3/roles` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:21.638208 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464d47d5-334e-4713-9ce7-cea9575f2b76 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:21.638733 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464d47d5-334e-4713-9ce7-cea9575f2b76 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:21.640405 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464d47d5-334e-4713-9ce7-cea9575f2b76 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:21.659698 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 393/789] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Aug 24 17:28:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 394 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:28:21.665443 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464d47d5-334e-4713-9ce7-cea9575f2b76 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:21.713778 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:21.714082 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c263d37d-c7d1-4fdd-bea7-021d5b7779a8 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:21.745226 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93002544-5209-417a-9ed2-db2af31dcabd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:21.745839 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93002544-5209-417a-9ed2-db2af31dcabd None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:21.752831 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93002544-5209-417a-9ed2-db2af31dcabd None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:21.752831 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93002544-5209-417a-9ed2-db2af31dcabd None admin] PATH_INFO: `/v3/domains` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:21.752831 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93002544-5209-417a-9ed2-db2af31dcabd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:21.752831 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93002544-5209-417a-9ed2-db2af31dcabd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:21.752831 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93002544-5209-417a-9ed2-db2af31dcabd None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:21.761463 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 397/790] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 24 17:28:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:21.771826 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93002544-5209-417a-9ed2-db2af31dcabd None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:21.826430 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:21.871066 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464d47d5-334e-4713-9ce7-cea9575f2b76 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:21.889444 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93002544-5209-417a-9ed2-db2af31dcabd None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:21.907065 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f8276d-a34a-4998-9b0b-830311b954d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:21.907831 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f8276d-a34a-4998-9b0b-830311b954d9 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:21.908115 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f8276d-a34a-4998-9b0b-830311b954d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:21.908388 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f8276d-a34a-4998-9b0b-830311b954d9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:21.920291 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f8276d-a34a-4998-9b0b-830311b954d9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:21.921886 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f8276d-a34a-4998-9b0b-830311b954d9 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:21.933298 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956840de-8fc2-4ef7-9a24-4e3ebbfa458a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:21.934058 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956840de-8fc2-4ef7-9a24-4e3ebbfa458a None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:21.934497 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956840de-8fc2-4ef7-9a24-4e3ebbfa458a None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:21.934869 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956840de-8fc2-4ef7-9a24-4e3ebbfa458a None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:21.937682 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application [None req-92f8276d-a34a-4998-9b0b-830311b954d9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:28:21.937682 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:28:21.937682 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:28:21.937682 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:28:21.937682 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:28:21.937682 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:28:21.937682 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:28:21.937682 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:28:21.937682 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:28:21.937682 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:28:21.937682 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:28:21.937682 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:28:21.937682 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:28:21.937682 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:28:21.937682 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:28:21.937682 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:28:21.938593 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:28:21.938593 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:28:21.938593 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:28:21.938593 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:28:21.938593 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:28:21.938593 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:28:21.938593 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:28:21.938593 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application with Lock( Aug 24 17:28:21.938593 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:28:21.938593 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:28:21.938593 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:28:21.938593 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:28:21.938593 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:28:21.938593 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:28:21.938593 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:28:21.938593 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:28:21.938593 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:28:21.939446 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:28:21.939446 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:28:21.939446 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Aug 24 17:28:21.939446 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 394/791] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 17:28:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 17:28:21.940411 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd05302b-df23-487f-970d-4fccac34fdde None None] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:21.940882 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd05302b-df23-487f-970d-4fccac34fdde None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:21.940991 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd05302b-df23-487f-970d-4fccac34fdde None None] PATH_INFO: `/` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:21.943465 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956840de-8fc2-4ef7-9a24-4e3ebbfa458a None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:21.945178 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956840de-8fc2-4ef7-9a24-4e3ebbfa458a None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:21.946356 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 395/792] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:28:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:28:21.947215 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd05302b-df23-487f-970d-4fccac34fdde None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:22.030614 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:22.066995 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd05302b-df23-487f-970d-4fccac34fdde None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:22.110294 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application [None req-956840de-8fc2-4ef7-9a24-4e3ebbfa458a None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 17:28:22.110294 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:28:22.110294 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:28:22.110294 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:28:22.110294 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:28:22.110294 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:28:22.110294 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:28:22.110294 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:28:22.110294 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:28:22.110294 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:28:22.110294 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:28:22.110294 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:28:22.110294 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 17:28:22.110294 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 17:28:22.110294 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 17:28:22.110294 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 17:28:22.112010 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:28:22.112010 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:28:22.112010 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 17:28:22.112010 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:28:22.112010 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 17:28:22.112010 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:28:22.112010 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:28:22.112010 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:28:22.112010 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:28:22.112010 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:28:22.112010 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:28:22.112010 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application with Lock( Aug 24 17:28:22.112010 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:28:22.112010 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:28:22.112010 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:28:22.112010 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:28:22.112010 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:28:22.112010 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:28:22.114324 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:28:22.114324 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:28:22.114324 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 17:28:22.114324 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 17:28:22.114324 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 17:28:22.114324 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 17:28:22.114324 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 17:28:22.114324 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 17:28:22.114324 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 17:28:22.114324 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Aug 24 17:28:22.114324 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 398/793] 10.222.0.45 () {56 vars in 1117 bytes} [Wed Aug 24 17:28:21 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 344 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:28:22.115692 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c54c81c-4ca9-4b2f-83b3-a839e738c508 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:22.115857 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b172aa4-117c-4778-a32a-b5f8e88907e4 None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:22.116009 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c54c81c-4ca9-4b2f-83b3-a839e738c508 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:22.116181 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b172aa4-117c-4778-a32a-b5f8e88907e4 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:22.116323 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c54c81c-4ca9-4b2f-83b3-a839e738c508 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:22.116463 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b172aa4-117c-4778-a32a-b5f8e88907e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:22.116624 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c54c81c-4ca9-4b2f-83b3-a839e738c508 None admin] PATH_INFO: `/v3/domains` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:22.116624 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c54c81c-4ca9-4b2f-83b3-a839e738c508 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:22.116624 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c54c81c-4ca9-4b2f-83b3-a839e738c508 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:22.122465 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c54c81c-4ca9-4b2f-83b3-a839e738c508 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:22.137163 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 396/794] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 24 17:28:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:22.137163 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c54c81c-4ca9-4b2f-83b3-a839e738c508 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:22.144398 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-0b172aa4-117c-4778-a32a-b5f8e88907e4 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:28:22.145625 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-0b172aa4-117c-4778-a32a-b5f8e88907e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:28:22.216116 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 399/795] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:28:22 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:28:22.217067 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b172aa4-117c-4778-a32a-b5f8e88907e4 None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:22.242447 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:22.245029 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c54c81c-4ca9-4b2f-83b3-a839e738c508 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:22.275569 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:22.283656 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1dcc394-8f91-4176-87bc-54fe86f9ea91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:22.284509 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dcc394-8f91-4176-87bc-54fe86f9ea91 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:22.285020 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dcc394-8f91-4176-87bc-54fe86f9ea91 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:22.285507 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dcc394-8f91-4176-87bc-54fe86f9ea91 None admin] PATH_INFO: `/v3/users` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:22.286456 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1dcc394-8f91-4176-87bc-54fe86f9ea91 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:22.286959 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1dcc394-8f91-4176-87bc-54fe86f9ea91 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:22.288604 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1dcc394-8f91-4176-87bc-54fe86f9ea91 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:22.307197 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 397/796] 10.222.0.45 () {56 vars in 1149 bytes} [Wed Aug 24 17:28:22 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:22.312884 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1dcc394-8f91-4176-87bc-54fe86f9ea91 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:22.323973 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b172aa4-117c-4778-a32a-b5f8e88907e4 None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:22.360835 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314f70d7-3ec1-4740-9673-35716031a7a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:22.361468 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314f70d7-3ec1-4740-9673-35716031a7a7 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:22.361748 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314f70d7-3ec1-4740-9673-35716031a7a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:22.362023 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314f70d7-3ec1-4740-9673-35716031a7a7 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:22.369418 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314f70d7-3ec1-4740-9673-35716031a7a7 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:22.371127 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314f70d7-3ec1-4740-9673-35716031a7a7 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:22.382697 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application [None req-314f70d7-3ec1-4740-9673-35716031a7a7 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 17:28:22.382697 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:28:22.382697 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:28:22.382697 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:28:22.382697 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:28:22.382697 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:28:22.382697 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:28:22.382697 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:28:22.382697 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:28:22.382697 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:28:22.382697 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:28:22.382697 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:28:22.382697 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 17:28:22.382697 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 17:28:22.382697 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 17:28:22.382697 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 17:28:22.384068 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:28:22.384068 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:28:22.384068 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 17:28:22.384068 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:28:22.384068 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 17:28:22.384068 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:28:22.384068 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:28:22.384068 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:28:22.384068 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:28:22.384068 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:28:22.384068 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:28:22.384068 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application with Lock( Aug 24 17:28:22.384068 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:28:22.384068 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:28:22.384068 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:28:22.384068 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:28:22.384068 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:28:22.384068 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:28:22.385012 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:28:22.385012 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:28:22.385012 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 17:28:22.385012 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 17:28:22.385012 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 17:28:22.385012 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 17:28:22.385012 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 17:28:22.385012 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 17:28:22.385012 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 17:28:22.385012 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Aug 24 17:28:22.385012 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 400/797] 10.222.0.45 () {56 vars in 1096 bytes} [Wed Aug 24 17:28:22 2022] GET /identity/v3/users/swift => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:28:22.389771 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314f70d7-3ec1-4740-9673-35716031a7a7 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:22.424536 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1dcc394-8f91-4176-87bc-54fe86f9ea91 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:22.460127 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71240baf-fa43-4177-89cd-0ec2a3a2dd85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:22.461145 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71240baf-fa43-4177-89cd-0ec2a3a2dd85 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:22.461653 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71240baf-fa43-4177-89cd-0ec2a3a2dd85 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:22.462153 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71240baf-fa43-4177-89cd-0ec2a3a2dd85 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:22.473440 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71240baf-fa43-4177-89cd-0ec2a3a2dd85 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:22.475196 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71240baf-fa43-4177-89cd-0ec2a3a2dd85 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:22.478506 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:22.494718 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application [None req-71240baf-fa43-4177-89cd-0ec2a3a2dd85 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:28:22.494718 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:28:22.494718 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:28:22.494718 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:28:22.494718 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:28:22.494718 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:28:22.494718 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:28:22.494718 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:28:22.494718 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:28:22.494718 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:28:22.494718 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:28:22.494718 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:28:22.494718 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:28:22.494718 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:28:22.494718 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:28:22.494718 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:28:22.495728 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:28:22.495728 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:28:22.495728 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:28:22.495728 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:28:22.495728 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:28:22.495728 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:28:22.495728 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:28:22.495728 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application with Lock( Aug 24 17:28:22.495728 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:28:22.495728 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:28:22.495728 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:28:22.495728 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:28:22.495728 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:28:22.495728 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:28:22.495728 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:28:22.495728 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:28:22.495728 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:28:22.496739 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:28:22.496739 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:28:22.496739 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Aug 24 17:28:22.504594 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 398/798] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 17:28:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 17:28:22.504594 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71240baf-fa43-4177-89cd-0ec2a3a2dd85 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:22.511757 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314f70d7-3ec1-4740-9673-35716031a7a7 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:22.582173 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dee1844-df4d-439b-9ad1-cb2c48e4b16e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:22.582678 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:22.583049 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dee1844-df4d-439b-9ad1-cb2c48e4b16e None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:22.583323 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dee1844-df4d-439b-9ad1-cb2c48e4b16e None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:22.583603 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dee1844-df4d-439b-9ad1-cb2c48e4b16e None admin] PATH_INFO: `/v3/users` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:22.584318 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dee1844-df4d-439b-9ad1-cb2c48e4b16e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:22.584639 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dee1844-df4d-439b-9ad1-cb2c48e4b16e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:22.586052 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dee1844-df4d-439b-9ad1-cb2c48e4b16e None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:22.622462 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 401/799] 10.222.0.45 () {56 vars in 1135 bytes} [Wed Aug 24 17:28:22 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:22.627895 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dee1844-df4d-439b-9ad1-cb2c48e4b16e None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:22.671120 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71240baf-fa43-4177-89cd-0ec2a3a2dd85 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:22.707915 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4c60f9-0560-48ab-ba65-126c4198c81b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:22.708663 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4c60f9-0560-48ab-ba65-126c4198c81b None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:22.708948 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4c60f9-0560-48ab-ba65-126c4198c81b None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:22.709225 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4c60f9-0560-48ab-ba65-126c4198c81b None admin] PATH_INFO: `/v3/domains` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:22.709875 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4c60f9-0560-48ab-ba65-126c4198c81b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:22.710179 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4c60f9-0560-48ab-ba65-126c4198c81b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:22.711896 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4c60f9-0560-48ab-ba65-126c4198c81b None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:22.725368 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 399/800] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 24 17:28:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:28:22.726324 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4c60f9-0560-48ab-ba65-126c4198c81b None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:22.757005 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dee1844-df4d-439b-9ad1-cb2c48e4b16e None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:22.799720 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8f4caf-6ad8-4dcf-bbbc-f263a7e84ec3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:22.800116 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8f4caf-6ad8-4dcf-bbbc-f263a7e84ec3 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:22.800399 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8f4caf-6ad8-4dcf-bbbc-f263a7e84ec3 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:22.800710 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8f4caf-6ad8-4dcf-bbbc-f263a7e84ec3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:22.815218 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8f4caf-6ad8-4dcf-bbbc-f263a7e84ec3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:22.817526 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8f4caf-6ad8-4dcf-bbbc-f263a7e84ec3 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:22.819477 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:22.838069 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application [None req-8f8f4caf-6ad8-4dcf-bbbc-f263a7e84ec3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:28:22.838069 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:28:22.838069 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:28:22.838069 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:28:22.838069 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:28:22.838069 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:28:22.838069 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:28:22.838069 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:28:22.838069 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:28:22.838069 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:28:22.838069 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:28:22.838069 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:28:22.838069 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:28:22.838069 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:28:22.838069 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:28:22.838069 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:28:22.839602 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:28:22.839602 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:28:22.839602 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:28:22.839602 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:28:22.839602 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:28:22.839602 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:28:22.839602 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:28:22.839602 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application with Lock( Aug 24 17:28:22.839602 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:28:22.839602 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:28:22.839602 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:28:22.839602 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:28:22.839602 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:28:22.839602 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:28:22.839602 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:28:22.839602 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:28:22.839602 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:28:22.840503 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:28:22.840503 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:28:22.840503 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Aug 24 17:28:22.840503 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 402/801] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 17:28:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:28:22.841236 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8f4caf-6ad8-4dcf-bbbc-f263a7e84ec3 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:22.858886 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4c60f9-0560-48ab-ba65-126c4198c81b None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:22.861061 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:22.901799 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5fdca7-29a9-4337-9aa3-470061c2846f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:22.902947 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5fdca7-29a9-4337-9aa3-470061c2846f None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:22.903446 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5fdca7-29a9-4337-9aa3-470061c2846f None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:22.903914 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5fdca7-29a9-4337-9aa3-470061c2846f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:22.915704 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5fdca7-29a9-4337-9aa3-470061c2846f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:28:22.921180 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application [None req-8e5fdca7-29a9-4337-9aa3-470061c2846f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:28:22.921180 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:28:22.921180 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:28:22.921180 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:28:22.921180 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:28:22.921180 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:28:22.921180 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:28:22.921180 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:28:22.921180 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:28:22.921180 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:28:22.921180 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:28:22.921180 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:28:22.921180 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:28:22.921180 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:28:22.921180 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:28:22.921180 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:28:22.922082 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:28:22.922082 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:28:22.922082 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:28:22.922082 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:28:22.922082 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:28:22.922082 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:28:22.922082 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:28:22.922082 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:28:22.922082 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:28:22.922082 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:28:22.922082 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:28:22.922082 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application with Lock( Aug 24 17:28:22.922082 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:28:22.922082 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:28:22.922082 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:28:22.922082 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:28:22.922927 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:28:22.922927 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:28:22.922927 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:28:22.922927 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:28:22.922927 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:28:22.922927 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:28:22.922927 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:28:22.922927 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:28:22.922927 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:28:22.922927 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:28:22.922927 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:28:22.922927 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Aug 24 17:28:22.928809 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 400/802] 10.222.0.45 () {56 vars in 1096 bytes} [Wed Aug 24 17:28:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:28:22.930007 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5fdca7-29a9-4337-9aa3-470061c2846f None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:22.956890 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8f4caf-6ad8-4dcf-bbbc-f263a7e84ec3 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:22.991548 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9970b0d-2220-425b-ba7b-16e1201e852a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:22.992175 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9970b0d-2220-425b-ba7b-16e1201e852a None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:22.992454 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9970b0d-2220-425b-ba7b-16e1201e852a None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:22.992751 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9970b0d-2220-425b-ba7b-16e1201e852a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:22.999773 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9970b0d-2220-425b-ba7b-16e1201e852a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:23.001238 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9970b0d-2220-425b-ba7b-16e1201e852a None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:23.012520 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application [None req-e9970b0d-2220-425b-ba7b-16e1201e852a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:28:23.012520 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:28:23.012520 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:28:23.012520 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:28:23.012520 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:28:23.012520 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:28:23.012520 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:28:23.012520 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:28:23.012520 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:28:23.012520 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:28:23.012520 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:28:23.012520 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:28:23.012520 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 17:28:23.012520 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 17:28:23.012520 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 17:28:23.013411 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 17:28:23.013411 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:28:23.013411 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:28:23.013411 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:28:23.013411 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:28:23.013411 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:28:23.013411 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:28:23.013411 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:28:23.013411 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application with Lock( Aug 24 17:28:23.013411 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:28:23.013411 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:28:23.013411 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:28:23.013411 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:28:23.013411 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:28:23.013411 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:28:23.013411 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:28:23.013411 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:28:23.014293 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 17:28:23.014293 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 17:28:23.014293 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 17:28:23.014293 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 17:28:23.014293 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 17:28:23.014293 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 17:28:23.014293 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:28:23.014293 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Aug 24 17:28:23.014293 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 403/803] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Aug 24 17:28:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:28:23.014781 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9970b0d-2220-425b-ba7b-16e1201e852a None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:23.038262 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5fdca7-29a9-4337-9aa3-470061c2846f None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:23.070499 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:23.071711 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb03144d-2ae8-435c-be67-b57ac2ef094a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:23.072671 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb03144d-2ae8-435c-be67-b57ac2ef094a None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:23.073544 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb03144d-2ae8-435c-be67-b57ac2ef094a None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:23.073808 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb03144d-2ae8-435c-be67-b57ac2ef094a None admin] PATH_INFO: `/v3/domains` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:23.074502 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb03144d-2ae8-435c-be67-b57ac2ef094a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:23.074739 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb03144d-2ae8-435c-be67-b57ac2ef094a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:23.076107 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb03144d-2ae8-435c-be67-b57ac2ef094a None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:23.087175 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 401/804] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 24 17:28:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:23.088411 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f93673e-8fab-45dd-8e6a-9f0f5f886a56 None None] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:23.088666 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f93673e-8fab-45dd-8e6a-9f0f5f886a56 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:23.088939 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f93673e-8fab-45dd-8e6a-9f0f5f886a56 None None] PATH_INFO: `/` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:23.092621 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 402/805] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:28:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:28:23.092621 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd391f2-2141-4f64-b86d-3119795656d7 None None] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:23.092621 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd391f2-2141-4f64-b86d-3119795656d7 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:23.092621 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd391f2-2141-4f64-b86d-3119795656d7 None None] PATH_INFO: `/` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:23.096731 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 403/806] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:28:23 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:28:23.096731 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd391f2-2141-4f64-b86d-3119795656d7 None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:23.118529 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:23.118747 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9970b0d-2220-425b-ba7b-16e1201e852a None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:23.153300 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a32bab5-f8ad-492c-93c7-9be96e9ea58d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:23.154003 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a32bab5-f8ad-492c-93c7-9be96e9ea58d None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:23.154311 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a32bab5-f8ad-492c-93c7-9be96e9ea58d None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:23.154930 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a32bab5-f8ad-492c-93c7-9be96e9ea58d None admin] PATH_INFO: `/v3/roles` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:23.155761 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a32bab5-f8ad-492c-93c7-9be96e9ea58d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:23.156391 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a32bab5-f8ad-492c-93c7-9be96e9ea58d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:23.157998 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a32bab5-f8ad-492c-93c7-9be96e9ea58d None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:23.166561 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 404/807] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Aug 24 17:28:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:23.167345 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a32bab5-f8ad-492c-93c7-9be96e9ea58d None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:23.200152 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd391f2-2141-4f64-b86d-3119795656d7 None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:23.233072 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4054f1-9ec8-4ed0-9360-cd2761a60de4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:23.233674 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4054f1-9ec8-4ed0-9360-cd2761a60de4 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:23.233946 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4054f1-9ec8-4ed0-9360-cd2761a60de4 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:23.234211 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4054f1-9ec8-4ed0-9360-cd2761a60de4 None admin] PATH_INFO: `/v3/projects` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:23.234889 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4054f1-9ec8-4ed0-9360-cd2761a60de4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:23.235188 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4054f1-9ec8-4ed0-9360-cd2761a60de4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:23.236656 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4054f1-9ec8-4ed0-9360-cd2761a60de4 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:23.248672 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 404/808] 10.222.0.45 () {56 vars in 1148 bytes} [Wed Aug 24 17:28:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:23.250086 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af548048-4129-4e9e-84b6-391fc091292c None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:23.250369 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af548048-4129-4e9e-84b6-391fc091292c None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:23.250642 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af548048-4129-4e9e-84b6-391fc091292c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:23.266944 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a32bab5-f8ad-492c-93c7-9be96e9ea58d None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:23.267509 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-af548048-4129-4e9e-84b6-391fc091292c None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:28:23.268704 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-af548048-4129-4e9e-84b6-391fc091292c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:28:23.282675 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:23.301375 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b489e49-3bbe-4848-a877-02701d058ee8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:23.301998 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b489e49-3bbe-4848-a877-02701d058ee8 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:23.302281 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b489e49-3bbe-4848-a877-02701d058ee8 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:23.302562 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b489e49-3bbe-4848-a877-02701d058ee8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:23.309668 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b489e49-3bbe-4848-a877-02701d058ee8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:23.311194 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b489e49-3bbe-4848-a877-02701d058ee8 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:23.322076 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application [None req-7b489e49-3bbe-4848-a877-02701d058ee8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:28:23.322076 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:28:23.322076 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:28:23.322076 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:28:23.322076 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:28:23.322076 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:28:23.322076 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:28:23.322076 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:28:23.322076 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:28:23.322076 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:28:23.322076 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:28:23.322076 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:28:23.322076 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 17:28:23.322076 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 17:28:23.322076 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 17:28:23.323015 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 17:28:23.323015 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:28:23.323015 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:28:23.323015 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:28:23.323015 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:28:23.323015 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:28:23.323015 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:28:23.323015 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:28:23.323015 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application with Lock( Aug 24 17:28:23.323015 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:28:23.323015 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:28:23.323015 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:28:23.323015 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:28:23.323015 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:28:23.323015 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:28:23.323015 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:28:23.323015 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:28:23.323875 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 17:28:23.323875 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 17:28:23.323875 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 17:28:23.323875 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 17:28:23.323875 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 17:28:23.323875 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 17:28:23.323875 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:28:23.323875 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Aug 24 17:28:23.323875 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 405/809] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Aug 24 17:28:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:28:23.324445 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7085581-cb06-4abc-a5eb-53cff9f528fb None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:23.324755 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7085581-cb06-4abc-a5eb-53cff9f528fb None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:23.324945 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7085581-cb06-4abc-a5eb-53cff9f528fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:23.336129 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 405/810] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:28:23 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:28:23.337853 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af548048-4129-4e9e-84b6-391fc091292c None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:23.341973 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-b7085581-cb06-4abc-a5eb-53cff9f528fb None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:28:23.343080 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-b7085581-cb06-4abc-a5eb-53cff9f528fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:28:23.351888 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:23.411846 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 406/811] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:28:23 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:28:23.413357 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7085581-cb06-4abc-a5eb-53cff9f528fb None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:23.456836 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af548048-4129-4e9e-84b6-391fc091292c None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:23.501346 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e694298e-ede5-4f14-ab54-07c0ae212911 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:23.502813 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e694298e-ede5-4f14-ab54-07c0ae212911 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:23.503546 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e694298e-ede5-4f14-ab54-07c0ae212911 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:23.504031 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e694298e-ede5-4f14-ab54-07c0ae212911 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:23.514042 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7085581-cb06-4abc-a5eb-53cff9f528fb None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:23.521293 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e694298e-ede5-4f14-ab54-07c0ae212911 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:23.522446 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:23.524452 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e694298e-ede5-4f14-ab54-07c0ae212911 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:23.547726 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application [None req-e694298e-ede5-4f14-ab54-07c0ae212911 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:28:23.547726 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:28:23.547726 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:28:23.547726 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:28:23.547726 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:28:23.547726 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:28:23.547726 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:28:23.547726 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:28:23.547726 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:28:23.547726 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:28:23.547726 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:28:23.547726 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:28:23.547726 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:28:23.547726 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:28:23.547726 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:28:23.547726 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:28:23.548728 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:28:23.548728 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:28:23.548728 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:28:23.548728 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:28:23.548728 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:28:23.548728 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:28:23.548728 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:28:23.548728 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application with Lock( Aug 24 17:28:23.548728 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:28:23.548728 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:28:23.548728 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:28:23.548728 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:28:23.548728 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:28:23.548728 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:28:23.548728 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:28:23.548728 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:28:23.548728 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:28:23.549587 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:28:23.549587 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:28:23.549587 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Aug 24 17:28:23.554678 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 406/812] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 17:28:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:28:23.555606 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e694298e-ede5-4f14-ab54-07c0ae212911 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:23.557884 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35032d53-f8ee-4d92-b006-c1c1d4873e1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:23.559238 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35032d53-f8ee-4d92-b006-c1c1d4873e1d None admin] REQUEST_METHOD: `PUT` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:23.560748 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35032d53-f8ee-4d92-b006-c1c1d4873e1d None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:23.561238 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35032d53-f8ee-4d92-b006-c1c1d4873e1d None admin] PATH_INFO: `/v3/projects/2ad8a32bd9d2434097bd08b111914ca1/users/ea143d042ee94ae7a4544d90d18db7db/roles/ad6a81fb5d1941888577eb0ce9474882` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:23.579739 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35032d53-f8ee-4d92-b006-c1c1d4873e1d None admin] RBAC: Authorizing `identity:create_grant(project_id=2ad8a32bd9d2434097bd08b111914ca1, user_id=ea143d042ee94ae7a4544d90d18db7db, role_id=ad6a81fb5d1941888577eb0ce9474882)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:23.581949 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35032d53-f8ee-4d92-b006-c1c1d4873e1d None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:23.599368 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 407/813] 10.222.0.45 () {58 vars in 1439 bytes} [Wed Aug 24 17:28:23 2022] PUT /identity/v3/projects/2ad8a32bd9d2434097bd08b111914ca1/users/ea143d042ee94ae7a4544d90d18db7db/roles/ad6a81fb5d1941888577eb0ce9474882 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:28:23.600525 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35032d53-f8ee-4d92-b006-c1c1d4873e1d None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:23.627062 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:23.673863 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e694298e-ede5-4f14-ab54-07c0ae212911 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:23.712851 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9522bc1a-243e-4380-8509-a0778071b4f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:23.714082 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9522bc1a-243e-4380-8509-a0778071b4f8 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:23.714365 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9522bc1a-243e-4380-8509-a0778071b4f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:23.714645 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9522bc1a-243e-4380-8509-a0778071b4f8 None admin] PATH_INFO: `/v3/projects` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:23.715390 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9522bc1a-243e-4380-8509-a0778071b4f8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:23.715699 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9522bc1a-243e-4380-8509-a0778071b4f8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:23.717193 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9522bc1a-243e-4380-8509-a0778071b4f8 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:23.736284 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 407/814] 10.222.0.45 () {56 vars in 1148 bytes} [Wed Aug 24 17:28:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:23.736284 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9522bc1a-243e-4380-8509-a0778071b4f8 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:23.756388 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35032d53-f8ee-4d92-b006-c1c1d4873e1d None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:23.793161 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73df54fe-4d81-4ccd-86ae-71e91b1329b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:23.796921 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73df54fe-4d81-4ccd-86ae-71e91b1329b4 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:23.796921 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73df54fe-4d81-4ccd-86ae-71e91b1329b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:23.796921 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73df54fe-4d81-4ccd-86ae-71e91b1329b4 None admin] PATH_INFO: `/v3/domains` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:23.796921 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73df54fe-4d81-4ccd-86ae-71e91b1329b4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:23.796921 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73df54fe-4d81-4ccd-86ae-71e91b1329b4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:23.797745 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73df54fe-4d81-4ccd-86ae-71e91b1329b4 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:23.810922 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 408/815] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 24 17:28:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:23.811749 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73df54fe-4d81-4ccd-86ae-71e91b1329b4 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:23.833211 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:23.858615 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9522bc1a-243e-4380-8509-a0778071b4f8 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:23.883053 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:23.894368 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff72144-e82b-49b8-a37a-8f2d3011e3d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:23.895222 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff72144-e82b-49b8-a37a-8f2d3011e3d3 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:23.895503 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff72144-e82b-49b8-a37a-8f2d3011e3d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:23.895780 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff72144-e82b-49b8-a37a-8f2d3011e3d3 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:23.896371 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff72144-e82b-49b8-a37a-8f2d3011e3d3 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:23.897796 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff72144-e82b-49b8-a37a-8f2d3011e3d3 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:23.907953 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application [None req-eff72144-e82b-49b8-a37a-8f2d3011e3d3 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 17:28:23.907953 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:28:23.907953 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 17:28:23.907953 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 17:28:23.907953 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 17:28:23.907953 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 17:28:23.907953 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 17:28:23.907953 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:28:23.907953 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 17:28:23.907953 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Aug 24 17:28:23.907953 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 17:28:23.907953 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Aug 24 17:28:23.907953 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:28:23.907953 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:28:23.907953 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:28:23.907953 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:28:23.907953 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:28:23.909178 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:28:23.909178 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:28:23.909178 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:28:23.909178 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:28:23.909178 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:28:23.909178 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:28:23.909178 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 17:28:23.909178 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 17:28:23.909178 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 17:28:23.909178 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 17:28:23.909178 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:28:23.909178 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:28:23.909178 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:28:23.909178 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:28:23.909178 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:28:23.909178 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:28:23.910200 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:28:23.910200 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application with Lock( Aug 24 17:28:23.910200 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:28:23.910200 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:28:23.910200 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:28:23.910200 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:28:23.910200 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:28:23.910200 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:28:23.910200 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:28:23.910200 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:28:23.910200 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 17:28:23.910200 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:28:23.910200 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 17:28:23.910200 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Aug 24 17:28:23.912041 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 408/816] 10.222.0.45 () {56 vars in 1129 bytes} [Wed Aug 24 17:28:23 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:28:23.912759 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff72144-e82b-49b8-a37a-8f2d3011e3d3 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:23.932493 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73df54fe-4d81-4ccd-86ae-71e91b1329b4 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:23.969514 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aaa4bd8-58f8-412f-8180-52110d8476ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:23.970144 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaa4bd8-58f8-412f-8180-52110d8476ac None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:23.970552 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaa4bd8-58f8-412f-8180-52110d8476ac None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:23.970827 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaa4bd8-58f8-412f-8180-52110d8476ac None admin] PATH_INFO: `/v3/services/network` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:23.971423 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aaa4bd8-58f8-412f-8180-52110d8476ac None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:23.973046 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aaa4bd8-58f8-412f-8180-52110d8476ac None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:23.983090 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application [None req-2aaa4bd8-58f8-412f-8180-52110d8476ac None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 17:28:23.983090 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:28:23.983090 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 17:28:23.983090 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 17:28:23.983090 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 17:28:23.983090 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 17:28:23.983090 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 17:28:23.983090 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:28:23.983090 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 17:28:23.983090 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Aug 24 17:28:23.983090 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 17:28:23.983090 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Aug 24 17:28:23.983090 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:28:23.983090 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:28:23.983090 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:28:23.983090 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:28:23.983090 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:28:23.984056 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:28:23.984056 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:28:23.984056 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:28:23.984056 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:28:23.984056 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:28:23.984056 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:28:23.984056 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 17:28:23.984056 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 17:28:23.984056 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 17:28:23.984056 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 17:28:23.984056 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:28:23.984056 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:28:23.984056 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:28:23.984056 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:28:23.984056 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:28:23.984056 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:28:23.984913 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:28:23.984913 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application with Lock( Aug 24 17:28:23.984913 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:28:23.984913 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:28:23.984913 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:28:23.984913 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:28:23.984913 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:28:23.984913 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:28:23.984913 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:28:23.984913 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:28:23.984913 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 17:28:23.984913 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:28:23.984913 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 17:28:23.984913 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Aug 24 17:28:23.984913 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 409/817] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Aug 24 17:28:23 2022] GET /identity/v3/services/network => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:28:23.985686 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aaa4bd8-58f8-412f-8180-52110d8476ac None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:24.019586 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff72144-e82b-49b8-a37a-8f2d3011e3d3 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:24.053399 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83eeef7-a336-4dad-8bbe-a9c7f5c96101 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:24.054148 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83eeef7-a336-4dad-8bbe-a9c7f5c96101 None admin] REQUEST_METHOD: `PUT` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:24.054495 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83eeef7-a336-4dad-8bbe-a9c7f5c96101 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:24.054834 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83eeef7-a336-4dad-8bbe-a9c7f5c96101 None admin] PATH_INFO: `/v3/projects/2ad8a32bd9d2434097bd08b111914ca1/users/90f0772b64c84faaa689e7272fa97898/roles/e411ad9eb96e4e8d85b59b5b700ada7b` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:24.056841 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83eeef7-a336-4dad-8bbe-a9c7f5c96101 None admin] RBAC: Authorizing `identity:create_grant(project_id=2ad8a32bd9d2434097bd08b111914ca1, user_id=90f0772b64c84faaa689e7272fa97898, role_id=e411ad9eb96e4e8d85b59b5b700ada7b)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:24.058364 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83eeef7-a336-4dad-8bbe-a9c7f5c96101 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:24.071885 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 409/818] 10.222.0.45 () {58 vars in 1439 bytes} [Wed Aug 24 17:28:23 2022] PUT /identity/v3/projects/2ad8a32bd9d2434097bd08b111914ca1/users/90f0772b64c84faaa689e7272fa97898/roles/e411ad9eb96e4e8d85b59b5b700ada7b => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:28:24.073309 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83eeef7-a336-4dad-8bbe-a9c7f5c96101 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:24.083970 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:24.112908 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aaa4bd8-58f8-412f-8180-52110d8476ac None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:24.145565 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:24.163411 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf63e3c-d037-43de-9c96-ece113b57708 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:24.164633 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf63e3c-d037-43de-9c96-ece113b57708 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:24.165126 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf63e3c-d037-43de-9c96-ece113b57708 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:24.165616 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf63e3c-d037-43de-9c96-ece113b57708 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:24.175113 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf63e3c-d037-43de-9c96-ece113b57708 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:24.176981 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf63e3c-d037-43de-9c96-ece113b57708 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:24.190659 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application [None req-bdf63e3c-d037-43de-9c96-ece113b57708 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 17:28:24.190659 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:28:24.190659 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:28:24.190659 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:28:24.190659 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:28:24.190659 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:28:24.190659 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:28:24.190659 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:28:24.190659 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:28:24.190659 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:28:24.190659 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:28:24.190659 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:28:24.190659 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 17:28:24.190659 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 17:28:24.190659 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 17:28:24.190659 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 17:28:24.191663 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:28:24.191663 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:28:24.191663 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 17:28:24.191663 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:28:24.191663 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 17:28:24.191663 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:28:24.191663 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:28:24.191663 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:28:24.191663 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:28:24.191663 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:28:24.191663 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:28:24.191663 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application with Lock( Aug 24 17:28:24.191663 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:28:24.191663 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:28:24.191663 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:28:24.191663 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:28:24.191663 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:28:24.191663 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:28:24.193076 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:28:24.193076 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:28:24.193076 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 17:28:24.193076 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 17:28:24.193076 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 17:28:24.193076 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 17:28:24.193076 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 17:28:24.193076 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 17:28:24.193076 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 17:28:24.193076 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Aug 24 17:28:24.195488 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 410/819] 10.222.0.45 () {56 vars in 1093 bytes} [Wed Aug 24 17:28:23 2022] GET /identity/v3/users/nova => generated 82 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:28:24.196722 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf63e3c-d037-43de-9c96-ece113b57708 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:24.216786 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83eeef7-a336-4dad-8bbe-a9c7f5c96101 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:24.257339 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6740da1e-8876-479d-9ac8-6dc58fc627eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:24.258096 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6740da1e-8876-479d-9ac8-6dc58fc627eb None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:24.258476 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6740da1e-8876-479d-9ac8-6dc58fc627eb None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:24.258858 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6740da1e-8876-479d-9ac8-6dc58fc627eb None admin] PATH_INFO: `/v3/services` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:24.259498 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6740da1e-8876-479d-9ac8-6dc58fc627eb None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:24.259800 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6740da1e-8876-479d-9ac8-6dc58fc627eb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:24.261865 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6740da1e-8876-479d-9ac8-6dc58fc627eb None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:24.267643 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 410/820] 10.222.0.45 () {56 vars in 1124 bytes} [Wed Aug 24 17:28:24 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:24.268267 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6740da1e-8876-479d-9ac8-6dc58fc627eb None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:24.324138 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf63e3c-d037-43de-9c96-ece113b57708 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:24.360124 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6efe46-05e1-425c-9f6b-83f52b98cb2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:24.360776 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6efe46-05e1-425c-9f6b-83f52b98cb2a None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:24.361058 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6efe46-05e1-425c-9f6b-83f52b98cb2a None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:24.361368 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6efe46-05e1-425c-9f6b-83f52b98cb2a None admin] PATH_INFO: `/v3/services` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:24.362007 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6efe46-05e1-425c-9f6b-83f52b98cb2a None admin] RBAC: Adding query filter params (name=network) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:24.362343 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6efe46-05e1-425c-9f6b-83f52b98cb2a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:24.363882 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6efe46-05e1-425c-9f6b-83f52b98cb2a None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:24.369063 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 411/821] 10.222.0.45 () {56 vars in 1112 bytes} [Wed Aug 24 17:28:24 2022] GET /identity/v3/services?name=network => generated 124 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:24.374241 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6efe46-05e1-425c-9f6b-83f52b98cb2a None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:24.380210 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:24.406578 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6740da1e-8876-479d-9ac8-6dc58fc627eb None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:24.424870 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:24.455843 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ec6271-ea5d-4794-a68d-55786c1330da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:24.456726 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ec6271-ea5d-4794-a68d-55786c1330da None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:24.457176 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ec6271-ea5d-4794-a68d-55786c1330da None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:24.457591 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ec6271-ea5d-4794-a68d-55786c1330da None admin] PATH_INFO: `/v3/users` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:24.458457 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ec6271-ea5d-4794-a68d-55786c1330da None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:24.458902 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ec6271-ea5d-4794-a68d-55786c1330da None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:24.460963 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ec6271-ea5d-4794-a68d-55786c1330da None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:24.485805 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 411/822] 10.222.0.45 () {56 vars in 1133 bytes} [Wed Aug 24 17:28:24 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:24.486249 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6efe46-05e1-425c-9f6b-83f52b98cb2a None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:24.487352 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ec6271-ea5d-4794-a68d-55786c1330da None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:24.528500 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20987e53-a494-4a1a-8090-f23228c188d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:24.529751 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20987e53-a494-4a1a-8090-f23228c188d6 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:24.531249 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20987e53-a494-4a1a-8090-f23228c188d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:24.531679 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20987e53-a494-4a1a-8090-f23228c188d6 None admin] PATH_INFO: `/v3/services` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:24.532618 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20987e53-a494-4a1a-8090-f23228c188d6 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:24.533069 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20987e53-a494-4a1a-8090-f23228c188d6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:24.534909 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20987e53-a494-4a1a-8090-f23228c188d6 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:24.542208 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 412/823] 10.222.0.45 () {56 vars in 1124 bytes} [Wed Aug 24 17:28:24 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:24.543920 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20987e53-a494-4a1a-8090-f23228c188d6 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:24.609453 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ec6271-ea5d-4794-a68d-55786c1330da None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:24.629719 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:24.651264 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf653430-393f-47d9-a7c2-5b974b0f41ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:24.652240 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf653430-393f-47d9-a7c2-5b974b0f41ee None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:24.652523 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf653430-393f-47d9-a7c2-5b974b0f41ee None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:24.652824 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf653430-393f-47d9-a7c2-5b974b0f41ee None admin] PATH_INFO: `/v3/services` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:24.653764 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf653430-393f-47d9-a7c2-5b974b0f41ee None admin] RBAC: Adding query filter params (type=network) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:24.654067 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf653430-393f-47d9-a7c2-5b974b0f41ee None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:24.655626 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf653430-393f-47d9-a7c2-5b974b0f41ee None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:24.666142 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 412/824] 10.222.0.45 () {56 vars in 1112 bytes} [Wed Aug 24 17:28:24 2022] GET /identity/v3/services?type=network => generated 350 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:24.666142 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf653430-393f-47d9-a7c2-5b974b0f41ee None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:24.680076 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20987e53-a494-4a1a-8090-f23228c188d6 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:24.712739 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:24.718635 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a187ba80-4a6f-4797-b9b6-d876937c6e7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:24.719241 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a187ba80-4a6f-4797-b9b6-d876937c6e7c None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:24.719516 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a187ba80-4a6f-4797-b9b6-d876937c6e7c None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:24.719787 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a187ba80-4a6f-4797-b9b6-d876937c6e7c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:24.791827 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a187ba80-4a6f-4797-b9b6-d876937c6e7c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:24.793502 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a187ba80-4a6f-4797-b9b6-d876937c6e7c None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:24.816271 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf653430-393f-47d9-a7c2-5b974b0f41ee None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:24.850301 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896a106d-f467-4165-8e5a-07c6709945cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:24.850902 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896a106d-f467-4165-8e5a-07c6709945cc None admin] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:24.851176 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896a106d-f467-4165-8e5a-07c6709945cc None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:24.851445 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896a106d-f467-4165-8e5a-07c6709945cc None admin] PATH_INFO: `/v3/endpoints` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:24.852099 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896a106d-f467-4165-8e5a-07c6709945cc None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:24.853485 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896a106d-f467-4165-8e5a-07c6709945cc None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:24.880247 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 413/825] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 17:28:24 2022] POST /identity/v3/endpoints => generated 346 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:28:24.881124 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896a106d-f467-4165-8e5a-07c6709945cc None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:24.886629 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:24.944100 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 413/826] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Aug 24 17:28:24 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 17:28:24.951467 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a187ba80-4a6f-4797-b9b6-d876937c6e7c None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:25.032477 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:25.068583 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896a106d-f467-4165-8e5a-07c6709945cc None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:25.077031 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a187ba80-4a6f-4797-b9b6-d876937c6e7c None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:25.119293 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b947b5e8-8777-449e-ad0d-3aa2b94e37b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:25.120876 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b947b5e8-8777-449e-ad0d-3aa2b94e37b1 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:25.121606 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b947b5e8-8777-449e-ad0d-3aa2b94e37b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:25.122058 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b947b5e8-8777-449e-ad0d-3aa2b94e37b1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:25.135479 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b947b5e8-8777-449e-ad0d-3aa2b94e37b1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:25.137276 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b947b5e8-8777-449e-ad0d-3aa2b94e37b1 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:25.138758 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4e0cf8-3ffc-4c0a-8627-c4ff04347ac4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:25.139467 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4e0cf8-3ffc-4c0a-8627-c4ff04347ac4 None admin] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:25.139884 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4e0cf8-3ffc-4c0a-8627-c4ff04347ac4 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:25.140213 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4e0cf8-3ffc-4c0a-8627-c4ff04347ac4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:25.141240 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4e0cf8-3ffc-4c0a-8627-c4ff04347ac4 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:25.142513 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4e0cf8-3ffc-4c0a-8627-c4ff04347ac4 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:25.157157 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application [None req-b947b5e8-8777-449e-ad0d-3aa2b94e37b1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:28:25.157157 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:28:25.157157 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:28:25.157157 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:28:25.157157 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:28:25.157157 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:28:25.157157 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:28:25.157157 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:28:25.157157 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:28:25.157157 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:28:25.157157 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:28:25.157157 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:28:25.157157 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:28:25.157157 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:28:25.157157 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:28:25.157157 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:28:25.158329 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:28:25.158329 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:28:25.158329 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:28:25.158329 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:28:25.158329 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:28:25.158329 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:28:25.158329 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:28:25.158329 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application with Lock( Aug 24 17:28:25.158329 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:28:25.158329 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:28:25.158329 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:28:25.158329 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:28:25.158329 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:28:25.158329 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:28:25.158329 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:28:25.158329 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:28:25.158329 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:28:25.159256 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:28:25.159256 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:28:25.159256 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Aug 24 17:28:25.161748 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 414/827] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 17:28:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 17:28:25.173281 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 414/828] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 17:28:24 2022] POST /identity/v3/endpoints => generated 337 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:28:25.177145 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b947b5e8-8777-449e-ad0d-3aa2b94e37b1 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:25.201546 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:25.294348 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b947b5e8-8777-449e-ad0d-3aa2b94e37b1 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:25.339002 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cc1adf-adc6-4cc2-a1ee-d09192485460 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:25.339708 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cc1adf-adc6-4cc2-a1ee-d09192485460 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:25.340055 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cc1adf-adc6-4cc2-a1ee-d09192485460 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:25.340435 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cc1adf-adc6-4cc2-a1ee-d09192485460 None admin] PATH_INFO: `/v3/domains` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:25.341178 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85cc1adf-adc6-4cc2-a1ee-d09192485460 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:25.341572 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85cc1adf-adc6-4cc2-a1ee-d09192485460 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:25.342943 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85cc1adf-adc6-4cc2-a1ee-d09192485460 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:25.356286 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 415/829] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 24 17:28:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:28:25.361458 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4e0cf8-3ffc-4c0a-8627-c4ff04347ac4 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:25.471272 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4e0cf8-3ffc-4c0a-8627-c4ff04347ac4 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:25.509273 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13283197-6cd5-4800-99e5-29a57c6ef2fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:25.509911 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13283197-6cd5-4800-99e5-29a57c6ef2fc None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:25.510723 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13283197-6cd5-4800-99e5-29a57c6ef2fc None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:25.510993 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13283197-6cd5-4800-99e5-29a57c6ef2fc None admin] PATH_INFO: `/v3/projects/service` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:25.518136 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13283197-6cd5-4800-99e5-29a57c6ef2fc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:25.519581 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13283197-6cd5-4800-99e5-29a57c6ef2fc None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:25.533281 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application [None req-13283197-6cd5-4800-99e5-29a57c6ef2fc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:28:25.533281 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:28:25.533281 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:28:25.533281 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:28:25.533281 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:28:25.533281 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:28:25.533281 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:28:25.533281 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:28:25.533281 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:28:25.533281 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:28:25.533281 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:28:25.533281 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:28:25.533281 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 17:28:25.533281 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 17:28:25.533281 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 17:28:25.534222 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 17:28:25.534222 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:28:25.534222 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:28:25.534222 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:28:25.534222 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:28:25.534222 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:28:25.534222 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:28:25.534222 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:28:25.534222 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application with Lock( Aug 24 17:28:25.534222 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:28:25.534222 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:28:25.534222 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:28:25.534222 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:28:25.534222 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:28:25.534222 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:28:25.534222 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:28:25.534222 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:28:25.535106 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 17:28:25.535106 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 17:28:25.535106 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 17:28:25.535106 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 17:28:25.535106 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 17:28:25.535106 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 17:28:25.535106 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:28:25.535106 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Aug 24 17:28:25.536345 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 415/830] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Aug 24 17:28:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:28:25.540102 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cc1adf-adc6-4cc2-a1ee-d09192485460 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:25.616113 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e12e96d-055e-4526-81a6-fcbdebc30e65 None None] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:25.616513 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e12e96d-055e-4526-81a6-fcbdebc30e65 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:25.616909 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e12e96d-055e-4526-81a6-fcbdebc30e65 None None] PATH_INFO: `/` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:25.618602 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 416/831] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:28:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 17:28:25.623660 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95dc62c-baec-4362-93ee-62bc008fb514 None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:25.624028 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95dc62c-baec-4362-93ee-62bc008fb514 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:25.624376 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95dc62c-baec-4362-93ee-62bc008fb514 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:25.626088 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:25.630915 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:25.645129 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-b95dc62c-baec-4362-93ee-62bc008fb514 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:28:25.646658 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-b95dc62c-baec-4362-93ee-62bc008fb514 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:28:25.659394 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cc1adf-adc6-4cc2-a1ee-d09192485460 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:25.704904 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d948760c-2bc3-41ed-8047-ccd1f2b87815 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:25.705516 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d948760c-2bc3-41ed-8047-ccd1f2b87815 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:25.706253 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d948760c-2bc3-41ed-8047-ccd1f2b87815 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:25.706528 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d948760c-2bc3-41ed-8047-ccd1f2b87815 None admin] PATH_INFO: `/v3/projects` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:25.707634 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d948760c-2bc3-41ed-8047-ccd1f2b87815 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:25.707945 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d948760c-2bc3-41ed-8047-ccd1f2b87815 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:25.709704 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d948760c-2bc3-41ed-8047-ccd1f2b87815 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:25.726572 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 416/832] 10.222.0.45 () {56 vars in 1148 bytes} [Wed Aug 24 17:28:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:25.732604 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d948760c-2bc3-41ed-8047-ccd1f2b87815 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:25.739858 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 417/833] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:28:25 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:28:25.837073 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d948760c-2bc3-41ed-8047-ccd1f2b87815 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:25.871557 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7074d3-1494-4012-9b72-cb5f5741de29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:25.872160 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7074d3-1494-4012-9b72-cb5f5741de29 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:25.872434 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7074d3-1494-4012-9b72-cb5f5741de29 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:25.872745 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7074d3-1494-4012-9b72-cb5f5741de29 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:25.873526 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7074d3-1494-4012-9b72-cb5f5741de29 None admin] RBAC: Adding query filter params (role.id=e411ad9eb96e4e8d85b59b5b700ada7b, scope.project.id=2ad8a32bd9d2434097bd08b111914ca1, user.id=053b7ce6d6f649ed91a26e434dab805e) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:25.873836 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7074d3-1494-4012-9b72-cb5f5741de29 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:25.875303 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7074d3-1494-4012-9b72-cb5f5741de29 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:25.878324 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:25.881861 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 417/834] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Aug 24 17:28:25 2022] GET /identity/v3/role_assignments?user.id=053b7ce6d6f649ed91a26e434dab805e&scope.project.id=2ad8a32bd9d2434097bd08b111914ca1&role.id=e411ad9eb96e4e8d85b59b5b700ada7b => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:28:26.028152 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95dc62c-baec-4362-93ee-62bc008fb514 None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:26.132707 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95dc62c-baec-4362-93ee-62bc008fb514 None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:26.167388 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:26.173766 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36eeb41-05b3-45a6-afca-4723b4a21411 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:26.174663 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36eeb41-05b3-45a6-afca-4723b4a21411 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:26.175161 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36eeb41-05b3-45a6-afca-4723b4a21411 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:26.175682 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36eeb41-05b3-45a6-afca-4723b4a21411 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:26.186826 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36eeb41-05b3-45a6-afca-4723b4a21411 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:28:26.191775 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application [None req-e36eeb41-05b3-45a6-afca-4723b4a21411 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 17:28:26.191775 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:28:26.191775 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:28:26.191775 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:28:26.191775 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:28:26.191775 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:28:26.191775 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:28:26.191775 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:28:26.191775 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:28:26.191775 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:28:26.191775 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:28:26.191775 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:28:26.191775 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:28:26.191775 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:28:26.191775 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:28:26.191775 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:28:26.192707 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:28:26.192707 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:28:26.192707 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:28:26.192707 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:28:26.192707 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:28:26.192707 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:28:26.192707 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:28:26.192707 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:28:26.192707 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:28:26.192707 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:28:26.192707 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:28:26.192707 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application with Lock( Aug 24 17:28:26.192707 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:28:26.192707 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:28:26.192707 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:28:26.192707 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:28:26.193524 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:28:26.193524 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:28:26.193524 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:28:26.193524 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:28:26.193524 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:28:26.193524 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:28:26.193524 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:28:26.193524 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:28:26.193524 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:28:26.193524 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:28:26.193524 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 17:28:26.193524 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Aug 24 17:28:26.195520 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f85f548-772d-46df-ab4e-238d34c748f7 None None] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:26.195809 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f85f548-772d-46df-ab4e-238d34c748f7 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:26.196081 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f85f548-772d-46df-ab4e-238d34c748f7 None None] PATH_INFO: `/` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:26.196783 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 418/835] 10.222.0.45 () {56 vars in 1102 bytes} [Wed Aug 24 17:28:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:28:26.200572 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 418/836] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:28:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:28:26.203208 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f85f548-772d-46df-ab4e-238d34c748f7 None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:26.203899 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1487251a-84a2-493a-926e-76442e62e519 None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:26.204417 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1487251a-84a2-493a-926e-76442e62e519 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:26.204925 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1487251a-84a2-493a-926e-76442e62e519 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:26.223294 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-1487251a-84a2-493a-926e-76442e62e519 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:28:26.224824 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-1487251a-84a2-493a-926e-76442e62e519 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:28:26.308663 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 419/837] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:28:26 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:28:26.314245 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f85f548-772d-46df-ab4e-238d34c748f7 None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:26.351068 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec109b72-bc36-4874-903f-e22464eb86ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:26.351722 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec109b72-bc36-4874-903f-e22464eb86ad None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:26.351998 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec109b72-bc36-4874-903f-e22464eb86ad None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:26.352314 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec109b72-bc36-4874-903f-e22464eb86ad None admin] PATH_INFO: `/v3/roles` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:26.353011 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec109b72-bc36-4874-903f-e22464eb86ad None admin] RBAC: Adding query filter params (name=service) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:26.353308 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec109b72-bc36-4874-903f-e22464eb86ad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:26.355293 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec109b72-bc36-4874-903f-e22464eb86ad None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:26.365337 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 419/838] 10.222.0.45 () {56 vars in 1103 bytes} [Wed Aug 24 17:28:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:26.371488 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1487251a-84a2-493a-926e-76442e62e519 None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:26.477944 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1487251a-84a2-493a-926e-76442e62e519 None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:26.506002 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:26.518231 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bce673-4e90-4851-a1de-3a13228b5333 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:26.518856 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bce673-4e90-4851-a1de-3a13228b5333 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:26.519134 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bce673-4e90-4851-a1de-3a13228b5333 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:26.519403 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bce673-4e90-4851-a1de-3a13228b5333 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:26.530453 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bce673-4e90-4851-a1de-3a13228b5333 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:26.531898 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bce673-4e90-4851-a1de-3a13228b5333 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:26.546441 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application [None req-b2bce673-4e90-4851-a1de-3a13228b5333 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:28:26.546441 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:28:26.546441 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:28:26.546441 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:28:26.546441 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:28:26.546441 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:28:26.546441 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:28:26.546441 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:28:26.546441 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:28:26.546441 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:28:26.546441 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:28:26.546441 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:28:26.546441 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:28:26.546441 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:28:26.546441 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:28:26.546441 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:28:26.547383 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:28:26.547383 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:28:26.547383 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:28:26.547383 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:28:26.547383 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:28:26.547383 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:28:26.547383 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:28:26.547383 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application with Lock( Aug 24 17:28:26.547383 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:28:26.547383 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:28:26.547383 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:28:26.547383 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:28:26.547383 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:28:26.547383 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:28:26.547383 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:28:26.547383 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:28:26.547383 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:28:26.548250 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:28:26.548250 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:28:26.548250 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Aug 24 17:28:26.548250 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 420/839] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 17:28:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 17:28:26.553212 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec109b72-bc36-4874-903f-e22464eb86ad None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:26.584186 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bce673-4e90-4851-a1de-3a13228b5333 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:26.621728 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:26.664527 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec109b72-bc36-4874-903f-e22464eb86ad None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:26.691662 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bce673-4e90-4851-a1de-3a13228b5333 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:26.699835 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e198d08d-3621-4e61-87a7-d0dd36891d66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:26.700655 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e198d08d-3621-4e61-87a7-d0dd36891d66 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:26.701100 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e198d08d-3621-4e61-87a7-d0dd36891d66 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:26.701526 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e198d08d-3621-4e61-87a7-d0dd36891d66 None admin] PATH_INFO: `/v3/domains` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:26.702302 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e198d08d-3621-4e61-87a7-d0dd36891d66 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:26.702802 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e198d08d-3621-4e61-87a7-d0dd36891d66 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:26.704370 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e198d08d-3621-4e61-87a7-d0dd36891d66 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:26.717084 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 420/840] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 24 17:28:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:26.724602 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e198d08d-3621-4e61-87a7-d0dd36891d66 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:26.733056 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402993e2-7d7f-47da-ab74-e6b221f60714 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:26.733921 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402993e2-7d7f-47da-ab74-e6b221f60714 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:26.734788 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402993e2-7d7f-47da-ab74-e6b221f60714 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:26.735227 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402993e2-7d7f-47da-ab74-e6b221f60714 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:26.749613 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402993e2-7d7f-47da-ab74-e6b221f60714 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:28:26.757081 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application [None req-402993e2-7d7f-47da-ab74-e6b221f60714 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:28:26.757081 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:28:26.757081 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:28:26.757081 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:28:26.757081 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:28:26.757081 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:28:26.757081 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:28:26.757081 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:28:26.757081 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:28:26.757081 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:28:26.757081 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:28:26.757081 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:28:26.757081 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:28:26.757081 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:28:26.757081 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:28:26.757081 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:28:26.758237 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:28:26.758237 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:28:26.758237 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:28:26.758237 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:28:26.758237 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:28:26.758237 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:28:26.758237 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:28:26.758237 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:28:26.758237 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:28:26.758237 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:28:26.758237 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:28:26.758237 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application with Lock( Aug 24 17:28:26.758237 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:28:26.758237 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:28:26.758237 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:28:26.758237 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:28:26.759448 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:28:26.759448 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:28:26.759448 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:28:26.759448 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:28:26.759448 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:28:26.759448 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:28:26.759448 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:28:26.759448 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:28:26.759448 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:28:26.759448 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:28:26.759448 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:28:26.759448 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Aug 24 17:28:26.768298 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 421/841] 10.222.0.45 () {56 vars in 1096 bytes} [Wed Aug 24 17:28:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 17:28:26.769334 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402993e2-7d7f-47da-ab74-e6b221f60714 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:26.818530 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:26.855025 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e198d08d-3621-4e61-87a7-d0dd36891d66 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:26.885536 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:26.890406 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402993e2-7d7f-47da-ab74-e6b221f60714 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:26.904409 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b9e3b1-7c4f-4747-a6d1-74929c1da910 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:26.905285 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b9e3b1-7c4f-4747-a6d1-74929c1da910 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:26.905563 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b9e3b1-7c4f-4747-a6d1-74929c1da910 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:26.905831 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b9e3b1-7c4f-4747-a6d1-74929c1da910 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:26.918290 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b9e3b1-7c4f-4747-a6d1-74929c1da910 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:26.920008 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b9e3b1-7c4f-4747-a6d1-74929c1da910 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:26.934473 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ccdf3a-3756-4291-9ac8-6f35626fdbc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:26.935266 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ccdf3a-3756-4291-9ac8-6f35626fdbc2 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:26.935613 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ccdf3a-3756-4291-9ac8-6f35626fdbc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:26.935956 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ccdf3a-3756-4291-9ac8-6f35626fdbc2 None admin] PATH_INFO: `/v3/roles` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:26.936454 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application [None req-f2b9e3b1-7c4f-4747-a6d1-74929c1da910 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 17:28:26.936454 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:28:26.936454 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:28:26.936454 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:28:26.936454 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:28:26.936454 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:28:26.936454 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:28:26.936454 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:28:26.936454 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:28:26.936454 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:28:26.936454 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:28:26.936454 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:28:26.936454 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 17:28:26.936454 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 17:28:26.936454 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 17:28:26.936454 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 17:28:26.937453 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:28:26.937453 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:28:26.937453 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 17:28:26.937453 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:28:26.937453 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 17:28:26.937453 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:28:26.937453 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:28:26.937453 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:28:26.937453 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:28:26.937453 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:28:26.937453 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:28:26.937453 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application with Lock( Aug 24 17:28:26.937453 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:28:26.937453 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:28:26.937453 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:28:26.937453 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:28:26.937453 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:28:26.937453 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:28:26.938370 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:28:26.938370 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:28:26.938370 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 17:28:26.938370 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 17:28:26.938370 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 17:28:26.938370 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 17:28:26.938370 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 17:28:26.938370 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 17:28:26.938370 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 17:28:26.938370 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Aug 24 17:28:26.938885 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ccdf3a-3756-4291-9ac8-6f35626fdbc2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:26.938885 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ccdf3a-3756-4291-9ac8-6f35626fdbc2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:26.938885 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ccdf3a-3756-4291-9ac8-6f35626fdbc2 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:26.941827 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 421/842] 10.222.0.45 () {56 vars in 1117 bytes} [Wed Aug 24 17:28:26 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 17:28:26.946728 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b9e3b1-7c4f-4747-a6d1-74929c1da910 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:26.956835 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 422/843] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Aug 24 17:28:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:26.957739 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ccdf3a-3756-4291-9ac8-6f35626fdbc2 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:27.058940 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:27.065975 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ccdf3a-3756-4291-9ac8-6f35626fdbc2 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:27.082897 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b9e3b1-7c4f-4747-a6d1-74929c1da910 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:27.105342 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d793e31a-525e-4890-adb1-340f6e89c8c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:27.106298 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d793e31a-525e-4890-adb1-340f6e89c8c0 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:27.106679 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d793e31a-525e-4890-adb1-340f6e89c8c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:27.107126 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d793e31a-525e-4890-adb1-340f6e89c8c0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:27.118366 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d793e31a-525e-4890-adb1-340f6e89c8c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:27.120035 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d793e31a-525e-4890-adb1-340f6e89c8c0 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:27.129043 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d9704d-a2d8-4131-b4fa-d7ae119e4382 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:27.130093 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d9704d-a2d8-4131-b4fa-d7ae119e4382 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:27.130515 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d9704d-a2d8-4131-b4fa-d7ae119e4382 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:27.130948 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d9704d-a2d8-4131-b4fa-d7ae119e4382 None admin] PATH_INFO: `/v3/users` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:27.131832 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d9704d-a2d8-4131-b4fa-d7ae119e4382 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:27.132277 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d9704d-a2d8-4131-b4fa-d7ae119e4382 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:27.133852 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d9704d-a2d8-4131-b4fa-d7ae119e4382 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:27.135915 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application [None req-d793e31a-525e-4890-adb1-340f6e89c8c0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:28:27.135915 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:28:27.135915 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:28:27.135915 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:28:27.135915 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:28:27.135915 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:28:27.135915 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:28:27.135915 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:28:27.135915 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:28:27.135915 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:28:27.135915 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:28:27.135915 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:28:27.135915 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:28:27.135915 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:28:27.135915 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:28:27.135915 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:28:27.137012 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:28:27.137012 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:28:27.137012 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:28:27.137012 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:28:27.137012 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:28:27.137012 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:28:27.137012 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:28:27.137012 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application with Lock( Aug 24 17:28:27.137012 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:28:27.137012 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:28:27.137012 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:28:27.137012 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:28:27.137012 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:28:27.137012 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:28:27.137012 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:28:27.137012 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:28:27.137012 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:28:27.138062 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:28:27.138062 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:28:27.138062 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Aug 24 17:28:27.144047 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 423/844] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 17:28:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:28:27.145069 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d793e31a-525e-4890-adb1-340f6e89c8c0 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:27.159381 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 422/845] 10.222.0.45 () {56 vars in 1149 bytes} [Wed Aug 24 17:28:26 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:28:27.164796 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d9704d-a2d8-4131-b4fa-d7ae119e4382 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:27.216147 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:27.255745 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d793e31a-525e-4890-adb1-340f6e89c8c0 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:27.274874 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d9704d-a2d8-4131-b4fa-d7ae119e4382 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:27.295701 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10403854-0a5d-4183-91a7-6c7dd74f7bb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:27.296589 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10403854-0a5d-4183-91a7-6c7dd74f7bb6 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:27.297099 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10403854-0a5d-4183-91a7-6c7dd74f7bb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:27.297562 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10403854-0a5d-4183-91a7-6c7dd74f7bb6 None admin] PATH_INFO: `/v3/domains` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:27.298308 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10403854-0a5d-4183-91a7-6c7dd74f7bb6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:27.298615 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10403854-0a5d-4183-91a7-6c7dd74f7bb6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:27.300001 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10403854-0a5d-4183-91a7-6c7dd74f7bb6 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:27.311759 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4cf332-926d-42c4-b07e-2a38aedffd5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:27.316452 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 424/846] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 24 17:28:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:27.316580 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4cf332-926d-42c4-b07e-2a38aedffd5a None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:27.316580 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4cf332-926d-42c4-b07e-2a38aedffd5a None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:27.316580 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4cf332-926d-42c4-b07e-2a38aedffd5a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:27.316800 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88537fa0-9f6f-41c9-9f25-e0653cf54487 None None] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:27.316800 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88537fa0-9f6f-41c9-9f25-e0653cf54487 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:27.316800 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88537fa0-9f6f-41c9-9f25-e0653cf54487 None None] PATH_INFO: `/` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:27.316800 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 425/847] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:28:27 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:28:27.319919 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88537fa0-9f6f-41c9-9f25-e0653cf54487 None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:27.331661 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:27.388286 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4cf332-926d-42c4-b07e-2a38aedffd5a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:27.389497 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4cf332-926d-42c4-b07e-2a38aedffd5a None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:27.431290 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88537fa0-9f6f-41c9-9f25-e0653cf54487 None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:27.433605 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:27.466740 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786e539f-f714-4bb4-be56-c5baee239612 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:27.467372 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786e539f-f714-4bb4-be56-c5baee239612 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:27.467648 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786e539f-f714-4bb4-be56-c5baee239612 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:27.467917 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786e539f-f714-4bb4-be56-c5baee239612 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:27.476081 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786e539f-f714-4bb4-be56-c5baee239612 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:27.477550 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786e539f-f714-4bb4-be56-c5baee239612 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:27.490655 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application [None req-786e539f-f714-4bb4-be56-c5baee239612 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 17:28:27.490655 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:28:27.490655 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:28:27.490655 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:28:27.490655 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:28:27.490655 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:28:27.490655 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:28:27.490655 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:28:27.490655 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:28:27.490655 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:28:27.490655 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:28:27.490655 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:28:27.490655 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 17:28:27.490655 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 17:28:27.490655 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 17:28:27.490655 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 17:28:27.492169 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:28:27.492169 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:28:27.492169 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 17:28:27.492169 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:28:27.492169 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 17:28:27.492169 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:28:27.492169 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:28:27.492169 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:28:27.492169 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:28:27.492169 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:28:27.492169 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:28:27.492169 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application with Lock( Aug 24 17:28:27.492169 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:28:27.492169 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:28:27.492169 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:28:27.492169 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:28:27.492169 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:28:27.492169 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:28:27.493329 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:28:27.493329 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:28:27.493329 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 17:28:27.493329 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 17:28:27.493329 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 17:28:27.493329 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 17:28:27.493329 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 17:28:27.493329 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 17:28:27.493329 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 17:28:27.493329 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Aug 24 17:28:27.496845 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 426/848] 10.222.0.45 () {56 vars in 1096 bytes} [Wed Aug 24 17:28:27 2022] GET /identity/v3/users/swift => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:28:27.498060 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5327ae50-d156-47fc-9e41-1b9a4d5b5e91 None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:27.498357 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5327ae50-d156-47fc-9e41-1b9a4d5b5e91 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:27.498621 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5327ae50-d156-47fc-9e41-1b9a4d5b5e91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:27.502006 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 423/849] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Aug 24 17:28:27 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 17:28:27.503013 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4cf332-926d-42c4-b07e-2a38aedffd5a None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:27.521303 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-5327ae50-d156-47fc-9e41-1b9a4d5b5e91 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:28:27.522777 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-5327ae50-d156-47fc-9e41-1b9a4d5b5e91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:28:27.588159 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:27.604836 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 427/850] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:28:27 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:28:27.605882 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5327ae50-d156-47fc-9e41-1b9a4d5b5e91 None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:27.608344 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4cf332-926d-42c4-b07e-2a38aedffd5a None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:27.651476 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506f2d83-30c7-4848-bc99-1fe50f08693b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:27.652341 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506f2d83-30c7-4848-bc99-1fe50f08693b None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:27.652818 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506f2d83-30c7-4848-bc99-1fe50f08693b None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:27.653236 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506f2d83-30c7-4848-bc99-1fe50f08693b None admin] PATH_INFO: `/v3/users` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:27.654157 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506f2d83-30c7-4848-bc99-1fe50f08693b None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:27.654605 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506f2d83-30c7-4848-bc99-1fe50f08693b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:27.656713 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506f2d83-30c7-4848-bc99-1fe50f08693b None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:27.670179 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:27.689122 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 424/851] 10.222.0.45 () {56 vars in 1135 bytes} [Wed Aug 24 17:28:27 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:27.692599 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506f2d83-30c7-4848-bc99-1fe50f08693b None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:27.731918 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5327ae50-d156-47fc-9e41-1b9a4d5b5e91 None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:27.774443 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adb7d8c-7137-40de-bebb-9aa2c850a59e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:27.775751 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adb7d8c-7137-40de-bebb-9aa2c850a59e None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:27.776235 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adb7d8c-7137-40de-bebb-9aa2c850a59e None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:27.776863 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adb7d8c-7137-40de-bebb-9aa2c850a59e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:27.790700 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adb7d8c-7137-40de-bebb-9aa2c850a59e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:27.793394 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adb7d8c-7137-40de-bebb-9aa2c850a59e None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:27.816440 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application [None req-7adb7d8c-7137-40de-bebb-9aa2c850a59e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:28:27.816440 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:28:27.816440 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:28:27.816440 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:28:27.816440 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:28:27.816440 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:28:27.816440 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:28:27.816440 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:28:27.816440 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:28:27.816440 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:28:27.816440 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:28:27.816440 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:28:27.816440 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:28:27.816440 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:28:27.816440 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:28:27.816440 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:28:27.817453 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:28:27.817453 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:28:27.817453 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:28:27.817453 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:28:27.817453 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:28:27.817453 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:28:27.817453 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:28:27.817453 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application with Lock( Aug 24 17:28:27.817453 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:28:27.817453 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:28:27.817453 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:28:27.817453 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:28:27.817453 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:28:27.817453 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:28:27.817453 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:28:27.817453 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:28:27.817453 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:28:27.818350 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:28:27.818350 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:28:27.818350 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Aug 24 17:28:27.818350 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 428/852] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 17:28:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:28:27.823050 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adb7d8c-7137-40de-bebb-9aa2c850a59e None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:27.828851 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506f2d83-30c7-4848-bc99-1fe50f08693b None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:27.876520 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b769f9bb-e68e-464f-9690-f149ca8bc433 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:27.877214 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b769f9bb-e68e-464f-9690-f149ca8bc433 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:27.877416 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b769f9bb-e68e-464f-9690-f149ca8bc433 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:27.877686 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b769f9bb-e68e-464f-9690-f149ca8bc433 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:27.883594 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:27.935701 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adb7d8c-7137-40de-bebb-9aa2c850a59e None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:27.946056 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:27.949890 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b769f9bb-e68e-464f-9690-f149ca8bc433 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:27.951457 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b769f9bb-e68e-464f-9690-f149ca8bc433 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:27.968290 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d52bb59-f730-4c4c-8818-9565b3e3c769 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:27.968935 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d52bb59-f730-4c4c-8818-9565b3e3c769 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:27.969204 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d52bb59-f730-4c4c-8818-9565b3e3c769 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:27.969472 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d52bb59-f730-4c4c-8818-9565b3e3c769 None admin] PATH_INFO: `/v3/domains` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:27.970109 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d52bb59-f730-4c4c-8818-9565b3e3c769 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:27.970410 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d52bb59-f730-4c4c-8818-9565b3e3c769 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:27.971769 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d52bb59-f730-4c4c-8818-9565b3e3c769 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:27.983776 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 429/853] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 24 17:28:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:28:27.984578 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d52bb59-f730-4c4c-8818-9565b3e3c769 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:28.048573 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 425/854] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Aug 24 17:28:27 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 17:28:28.049844 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4eac3e1-9c3a-4631-8af4-1fb27816e757 None None] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:28.050149 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4eac3e1-9c3a-4631-8af4-1fb27816e757 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:28.050409 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4eac3e1-9c3a-4631-8af4-1fb27816e757 None None] PATH_INFO: `/` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:28.055844 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 426/855] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:28:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:28:28.055844 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4eac3e1-9c3a-4631-8af4-1fb27816e757 None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:28.080160 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d52bb59-f730-4c4c-8818-9565b3e3c769 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:28.091227 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:28.115783 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb10cb74-cbe7-4bfd-89b7-3c80462db2b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:28.116407 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb10cb74-cbe7-4bfd-89b7-3c80462db2b5 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:28.116714 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb10cb74-cbe7-4bfd-89b7-3c80462db2b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:28.116989 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb10cb74-cbe7-4bfd-89b7-3c80462db2b5 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:28.117570 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb10cb74-cbe7-4bfd-89b7-3c80462db2b5 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:28.118939 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb10cb74-cbe7-4bfd-89b7-3c80462db2b5 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:28.127999 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application [None req-cb10cb74-cbe7-4bfd-89b7-3c80462db2b5 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 17:28:28.127999 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:28:28.127999 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 17:28:28.127999 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 17:28:28.127999 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 17:28:28.127999 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 17:28:28.127999 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 17:28:28.127999 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:28:28.127999 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 17:28:28.127999 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Aug 24 17:28:28.127999 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 17:28:28.127999 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Aug 24 17:28:28.127999 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:28:28.127999 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:28:28.127999 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:28:28.127999 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:28:28.127999 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:28:28.129193 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:28:28.129193 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:28:28.129193 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:28:28.129193 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:28:28.129193 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:28:28.129193 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:28:28.129193 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 17:28:28.129193 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 17:28:28.129193 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 17:28:28.129193 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 17:28:28.129193 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:28:28.129193 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:28:28.129193 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:28:28.129193 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:28:28.129193 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:28:28.129193 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:28:28.130002 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:28:28.130002 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application with Lock( Aug 24 17:28:28.130002 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:28:28.130002 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:28:28.130002 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:28:28.130002 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:28:28.130002 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:28:28.130002 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:28:28.130002 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:28:28.130002 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:28:28.130002 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 17:28:28.130002 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:28:28.130002 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 17:28:28.130002 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Aug 24 17:28:28.130002 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 430/856] 10.222.0.45 () {56 vars in 1114 bytes} [Wed Aug 24 17:28:27 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 145 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:28:28.130775 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb10cb74-cbe7-4bfd-89b7-3c80462db2b5 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:28.153743 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:28.160187 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4eac3e1-9c3a-4631-8af4-1fb27816e757 None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:28.194025 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93a675c-de4d-4b07-ac0f-cf8b3c40b22e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:28.194575 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93a675c-de4d-4b07-ac0f-cf8b3c40b22e None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:28.194850 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93a675c-de4d-4b07-ac0f-cf8b3c40b22e None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:28.195120 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93a675c-de4d-4b07-ac0f-cf8b3c40b22e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:28.201488 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93a675c-de4d-4b07-ac0f-cf8b3c40b22e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:28.202906 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93a675c-de4d-4b07-ac0f-cf8b3c40b22e None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:28.214232 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application [None req-d93a675c-de4d-4b07-ac0f-cf8b3c40b22e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:28:28.214232 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:28:28.214232 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:28:28.214232 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:28:28.214232 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:28:28.214232 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:28:28.214232 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:28:28.214232 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:28:28.214232 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:28:28.214232 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:28:28.214232 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:28:28.214232 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:28:28.214232 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 17:28:28.214232 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 17:28:28.214232 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 17:28:28.215270 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 17:28:28.215270 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:28:28.215270 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:28:28.215270 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:28:28.215270 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:28:28.215270 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:28:28.215270 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:28:28.215270 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:28:28.215270 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application with Lock( Aug 24 17:28:28.215270 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:28:28.215270 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:28:28.215270 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:28:28.215270 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:28:28.215270 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:28:28.215270 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:28:28.215270 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:28:28.215270 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:28:28.216147 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 17:28:28.216147 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 17:28:28.216147 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 17:28:28.216147 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 17:28:28.216147 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 17:28:28.216147 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 17:28:28.216147 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:28:28.216147 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Aug 24 17:28:28.216147 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 427/857] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Aug 24 17:28:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:28:28.216983 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de490da0-ac92-4b1b-9413-3535c5bbc979 None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:28.216983 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de490da0-ac92-4b1b-9413-3535c5bbc979 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:28.216983 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de490da0-ac92-4b1b-9413-3535c5bbc979 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:28.229575 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb10cb74-cbe7-4bfd-89b7-3c80462db2b5 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:28.235074 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-de490da0-ac92-4b1b-9413-3535c5bbc979 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:28:28.236261 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-de490da0-ac92-4b1b-9413-3535c5bbc979 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:28:28.262344 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2156a937-0151-40e3-9cda-f3021c4cdd09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:28.262970 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2156a937-0151-40e3-9cda-f3021c4cdd09 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:28.263247 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2156a937-0151-40e3-9cda-f3021c4cdd09 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:28.263518 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2156a937-0151-40e3-9cda-f3021c4cdd09 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:28.273775 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2156a937-0151-40e3-9cda-f3021c4cdd09 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:28.275215 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2156a937-0151-40e3-9cda-f3021c4cdd09 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:28.289664 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application [None req-2156a937-0151-40e3-9cda-f3021c4cdd09 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:28:28.289664 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:28:28.289664 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:28:28.289664 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:28:28.289664 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:28:28.289664 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:28:28.289664 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:28:28.289664 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:28:28.289664 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:28:28.289664 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:28:28.289664 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:28:28.289664 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:28:28.289664 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:28:28.289664 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:28:28.289664 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:28:28.289664 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:28:28.290620 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:28:28.290620 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:28:28.290620 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:28:28.290620 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:28:28.290620 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:28:28.290620 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:28:28.290620 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:28:28.290620 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application with Lock( Aug 24 17:28:28.290620 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:28:28.290620 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:28:28.290620 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:28:28.290620 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:28:28.290620 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:28:28.290620 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:28:28.290620 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:28:28.290620 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:28:28.290620 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:28:28.292456 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:28:28.292456 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:28:28.292456 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Aug 24 17:28:28.292456 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 431/858] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 17:28:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:28:28.292456 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2156a937-0151-40e3-9cda-f3021c4cdd09 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:28.302932 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 428/859] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:28:28 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:28:28.303752 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de490da0-ac92-4b1b-9413-3535c5bbc979 None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:28.325237 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:28.401676 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2156a937-0151-40e3-9cda-f3021c4cdd09 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:28.417769 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:28.431877 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de490da0-ac92-4b1b-9413-3535c5bbc979 None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:28.437717 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11da36d5-2a6c-40d1-beb0-91c5105499d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:28.438324 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11da36d5-2a6c-40d1-beb0-91c5105499d4 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:28.438600 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11da36d5-2a6c-40d1-beb0-91c5105499d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:28.438869 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11da36d5-2a6c-40d1-beb0-91c5105499d4 None admin] PATH_INFO: `/v3/services` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:28.439492 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11da36d5-2a6c-40d1-beb0-91c5105499d4 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:28.439792 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11da36d5-2a6c-40d1-beb0-91c5105499d4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:28.441182 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11da36d5-2a6c-40d1-beb0-91c5105499d4 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:28.445688 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 432/860] 10.222.0.45 () {56 vars in 1114 bytes} [Wed Aug 24 17:28:28 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:28.446399 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11da36d5-2a6c-40d1-beb0-91c5105499d4 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:28.475567 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc61db6-cfd9-456b-b014-f798c213205f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:28.476173 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc61db6-cfd9-456b-b014-f798c213205f None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:28.476452 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc61db6-cfd9-456b-b014-f798c213205f None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:28.476762 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc61db6-cfd9-456b-b014-f798c213205f None admin] PATH_INFO: `/v3/projects` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:28.477451 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc61db6-cfd9-456b-b014-f798c213205f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:28.477755 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc61db6-cfd9-456b-b014-f798c213205f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:28.479721 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc61db6-cfd9-456b-b014-f798c213205f None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:28.496874 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 429/861] 10.222.0.45 () {56 vars in 1148 bytes} [Wed Aug 24 17:28:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:28.498570 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc61db6-cfd9-456b-b014-f798c213205f None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:28.545784 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:28.555135 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11da36d5-2a6c-40d1-beb0-91c5105499d4 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:28.589542 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1928522-e8fb-46be-8c78-55bea2c10501 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:28.590145 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1928522-e8fb-46be-8c78-55bea2c10501 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:28.590419 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1928522-e8fb-46be-8c78-55bea2c10501 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:28.590686 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1928522-e8fb-46be-8c78-55bea2c10501 None admin] PATH_INFO: `/v3/domains` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:28.591323 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1928522-e8fb-46be-8c78-55bea2c10501 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:28.591621 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1928522-e8fb-46be-8c78-55bea2c10501 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:28.593015 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1928522-e8fb-46be-8c78-55bea2c10501 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:28.603990 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc61db6-cfd9-456b-b014-f798c213205f None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:28.604396 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 433/862] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 24 17:28:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:28.605211 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1928522-e8fb-46be-8c78-55bea2c10501 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:28.637112 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4aeced-3126-43b2-90d1-4962510a460e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:28.637743 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4aeced-3126-43b2-90d1-4962510a460e None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:28.638024 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4aeced-3126-43b2-90d1-4962510a460e None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:28.638295 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4aeced-3126-43b2-90d1-4962510a460e None admin] PATH_INFO: `/v3/services` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:28.638920 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4aeced-3126-43b2-90d1-4962510a460e None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:28.639223 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4aeced-3126-43b2-90d1-4962510a460e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:28.640608 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4aeced-3126-43b2-90d1-4962510a460e None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:28.644851 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 430/863] 10.222.0.45 () {56 vars in 1114 bytes} [Wed Aug 24 17:28:28 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:28.645614 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4aeced-3126-43b2-90d1-4962510a460e None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:28.661389 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:28.731809 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1928522-e8fb-46be-8c78-55bea2c10501 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:28.761560 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4aeced-3126-43b2-90d1-4962510a460e None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:28.774984 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf17ed4-a79a-40b9-9923-d34a0fcb2bba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:28.775624 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf17ed4-a79a-40b9-9923-d34a0fcb2bba None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:28.776605 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf17ed4-a79a-40b9-9923-d34a0fcb2bba None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:28.776822 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf17ed4-a79a-40b9-9923-d34a0fcb2bba None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:28.777630 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf17ed4-a79a-40b9-9923-d34a0fcb2bba None admin] RBAC: Adding query filter params (role.id=ad6a81fb5d1941888577eb0ce9474882, scope.project.id=2ad8a32bd9d2434097bd08b111914ca1, user.id=ea143d042ee94ae7a4544d90d18db7db) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:28.777952 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf17ed4-a79a-40b9-9923-d34a0fcb2bba None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:28.779467 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf17ed4-a79a-40b9-9923-d34a0fcb2bba None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:28.788680 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 434/864] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Aug 24 17:28:28 2022] GET /identity/v3/role_assignments?user.id=ea143d042ee94ae7a4544d90d18db7db&scope.project.id=2ad8a32bd9d2434097bd08b111914ca1&role.id=ad6a81fb5d1941888577eb0ce9474882 => generated 608 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:28.788680 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf17ed4-a79a-40b9-9923-d34a0fcb2bba None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:28.808594 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:28.818119 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f17cf0a-1e75-4295-aae1-028386968e31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:28.819140 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f17cf0a-1e75-4295-aae1-028386968e31 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:28.819862 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f17cf0a-1e75-4295-aae1-028386968e31 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:28.819862 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f17cf0a-1e75-4295-aae1-028386968e31 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:28.820292 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f17cf0a-1e75-4295-aae1-028386968e31 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:28.821787 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f17cf0a-1e75-4295-aae1-028386968e31 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:28.834794 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application [None req-9f17cf0a-1e75-4295-aae1-028386968e31 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 17:28:28.834794 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:28:28.834794 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 17:28:28.834794 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 17:28:28.834794 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 17:28:28.834794 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 17:28:28.834794 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 17:28:28.834794 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:28:28.834794 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 17:28:28.834794 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Aug 24 17:28:28.834794 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 17:28:28.834794 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Aug 24 17:28:28.834794 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:28:28.834794 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:28:28.834794 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:28:28.834794 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:28:28.835994 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:28:28.835994 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:28:28.835994 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:28:28.835994 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:28:28.835994 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:28:28.835994 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:28:28.835994 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:28:28.835994 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 17:28:28.835994 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 17:28:28.835994 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 17:28:28.835994 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 17:28:28.835994 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:28:28.835994 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:28:28.835994 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:28:28.835994 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:28:28.835994 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:28:28.835994 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:28:28.836899 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:28:28.836899 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application with Lock( Aug 24 17:28:28.836899 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:28:28.836899 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:28:28.836899 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:28:28.836899 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:28:28.836899 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:28:28.836899 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:28:28.836899 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:28:28.836899 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:28:28.836899 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 17:28:28.836899 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:28:28.836899 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 17:28:28.836899 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Aug 24 17:28:28.836899 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 431/865] 10.222.0.45 () {56 vars in 1132 bytes} [Wed Aug 24 17:28:28 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:28:28.842911 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f17cf0a-1e75-4295-aae1-028386968e31 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:28.916861 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf17ed4-a79a-40b9-9923-d34a0fcb2bba None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:28.928991 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:28.962741 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5d4bee-f97f-46ed-943f-ba988bfc7f84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:28.962975 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5d4bee-f97f-46ed-943f-ba988bfc7f84 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:28.963230 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5d4bee-f97f-46ed-943f-ba988bfc7f84 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:28.963363 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f17cf0a-1e75-4295-aae1-028386968e31 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:28.963520 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5d4bee-f97f-46ed-943f-ba988bfc7f84 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:28.971812 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5d4bee-f97f-46ed-943f-ba988bfc7f84 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:28.973344 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5d4bee-f97f-46ed-943f-ba988bfc7f84 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:28.985930 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application [None req-7e5d4bee-f97f-46ed-943f-ba988bfc7f84 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:28:28.985930 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:28:28.985930 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:28:28.985930 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:28:28.985930 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:28:28.985930 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:28:28.985930 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:28:28.985930 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:28:28.985930 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:28:28.985930 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:28:28.985930 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:28:28.985930 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:28:28.985930 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 17:28:28.985930 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 17:28:28.985930 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 17:28:28.986796 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 17:28:28.986796 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:28:28.986796 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:28:28.986796 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:28:28.986796 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:28:28.986796 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:28:28.986796 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:28:28.986796 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:28:28.986796 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application with Lock( Aug 24 17:28:28.986796 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:28:28.986796 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:28:28.986796 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:28:28.986796 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:28:28.986796 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:28:28.986796 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:28:28.986796 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:28:28.986796 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:28:28.987677 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 17:28:28.987677 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 17:28:28.987677 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 17:28:28.987677 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 17:28:28.987677 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 17:28:28.987677 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 17:28:28.987677 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:28:28.987677 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Aug 24 17:28:28.987677 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 435/866] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Aug 24 17:28:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:28:28.992793 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5d4bee-f97f-46ed-943f-ba988bfc7f84 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:29.007124 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c5d69f-b66a-4a1d-a32d-3332c3eb8f81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:29.007977 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c5d69f-b66a-4a1d-a32d-3332c3eb8f81 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:29.008419 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c5d69f-b66a-4a1d-a32d-3332c3eb8f81 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:29.008852 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c5d69f-b66a-4a1d-a32d-3332c3eb8f81 None admin] PATH_INFO: `/v3/services` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:29.009628 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c5d69f-b66a-4a1d-a32d-3332c3eb8f81 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:29.010132 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c5d69f-b66a-4a1d-a32d-3332c3eb8f81 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:29.012159 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c5d69f-b66a-4a1d-a32d-3332c3eb8f81 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:29.018172 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 432/867] 10.222.0.45 () {56 vars in 1126 bytes} [Wed Aug 24 17:28:28 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:28:29.023776 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c5d69f-b66a-4a1d-a32d-3332c3eb8f81 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:29.079767 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:29.120348 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5d4bee-f97f-46ed-943f-ba988bfc7f84 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:29.151180 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c5d69f-b66a-4a1d-a32d-3332c3eb8f81 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:29.164880 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ec933d-ed58-4439-a957-cc2d47abf4bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:29.165753 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ec933d-ed58-4439-a957-cc2d47abf4bd None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:29.166236 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ec933d-ed58-4439-a957-cc2d47abf4bd None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:29.166696 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ec933d-ed58-4439-a957-cc2d47abf4bd None admin] PATH_INFO: `/v3/projects` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:29.167587 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ec933d-ed58-4439-a957-cc2d47abf4bd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:29.168035 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ec933d-ed58-4439-a957-cc2d47abf4bd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:29.169651 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ec933d-ed58-4439-a957-cc2d47abf4bd None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:29.183075 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 436/868] 10.222.0.45 () {56 vars in 1148 bytes} [Wed Aug 24 17:28:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:28:29.189538 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ec933d-ed58-4439-a957-cc2d47abf4bd None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:29.196914 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ce6af5-3055-4fd6-8195-a79fdcdc233c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:29.198397 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ce6af5-3055-4fd6-8195-a79fdcdc233c None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:29.198913 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ce6af5-3055-4fd6-8195-a79fdcdc233c None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:29.199364 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ce6af5-3055-4fd6-8195-a79fdcdc233c None admin] PATH_INFO: `/v3/services` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:29.200283 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ce6af5-3055-4fd6-8195-a79fdcdc233c None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:29.200795 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ce6af5-3055-4fd6-8195-a79fdcdc233c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:29.202418 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ce6af5-3055-4fd6-8195-a79fdcdc233c None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:29.207721 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 433/869] 10.222.0.45 () {56 vars in 1126 bytes} [Wed Aug 24 17:28:29 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:28:29.304953 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ec933d-ed58-4439-a957-cc2d47abf4bd None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:29.344755 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff05029-3c9c-4bb4-a2c8-9e1e6fbbba19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:29.344755 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff05029-3c9c-4bb4-a2c8-9e1e6fbbba19 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:29.344755 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff05029-3c9c-4bb4-a2c8-9e1e6fbbba19 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:29.344755 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff05029-3c9c-4bb4-a2c8-9e1e6fbbba19 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:29.345362 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff05029-3c9c-4bb4-a2c8-9e1e6fbbba19 None admin] RBAC: Adding query filter params (role.id=e411ad9eb96e4e8d85b59b5b700ada7b, scope.project.id=2ad8a32bd9d2434097bd08b111914ca1, user.id=90f0772b64c84faaa689e7272fa97898) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:29.345560 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff05029-3c9c-4bb4-a2c8-9e1e6fbbba19 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:29.347234 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff05029-3c9c-4bb4-a2c8-9e1e6fbbba19 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:29.348868 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:29.356700 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 437/870] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Aug 24 17:28:29 2022] GET /identity/v3/role_assignments?user.id=90f0772b64c84faaa689e7272fa97898&scope.project.id=2ad8a32bd9d2434097bd08b111914ca1&role.id=e411ad9eb96e4e8d85b59b5b700ada7b => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:28:30.602854 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb68012f-e256-4570-8d9c-779ded19de52 None None] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:30.603381 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb68012f-e256-4570-8d9c-779ded19de52 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:30.603381 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb68012f-e256-4570-8d9c-779ded19de52 None None] PATH_INFO: `/` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:30.605032 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 434/871] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:28:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:28:30.610610 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12d934f-e065-4497-a0c1-989bdd186863 None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:30.611009 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12d934f-e065-4497-a0c1-989bdd186863 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:30.611202 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12d934f-e065-4497-a0c1-989bdd186863 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:30.629588 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-c12d934f-e065-4497-a0c1-989bdd186863 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:28:30.630768 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-c12d934f-e065-4497-a0c1-989bdd186863 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:28:30.697493 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 438/872] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:28:30 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 17:28:30.863868 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ff046f-14de-497f-8d48-74adf47c6030 None None] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:30.864061 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ff046f-14de-497f-8d48-74adf47c6030 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:30.864335 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ff046f-14de-497f-8d48-74adf47c6030 None None] PATH_INFO: `/` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:30.865985 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 435/873] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:28:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:28:30.871766 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a9155c-6fed-49c5-95e9-4a26f4924fd3 None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:30.872061 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a9155c-6fed-49c5-95e9-4a26f4924fd3 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:30.872334 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a9155c-6fed-49c5-95e9-4a26f4924fd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:30.890393 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-c0a9155c-6fed-49c5-95e9-4a26f4924fd3 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:28:30.891580 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-c0a9155c-6fed-49c5-95e9-4a26f4924fd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:28:30.961484 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 439/874] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:28:30 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:28:30.969722 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ff046f-14de-497f-8d48-74adf47c6030 None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:30.974588 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:31.078099 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ff046f-14de-497f-8d48-74adf47c6030 None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:31.113000 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcc9e85-7dcb-4f27-9da5-76b6974f8566 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:31.113610 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcc9e85-7dcb-4f27-9da5-76b6974f8566 None admin] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:31.113945 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcc9e85-7dcb-4f27-9da5-76b6974f8566 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:31.114166 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcc9e85-7dcb-4f27-9da5-76b6974f8566 None admin] PATH_INFO: `/v3/services` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:31.114830 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcc9e85-7dcb-4f27-9da5-76b6974f8566 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:31.116261 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcc9e85-7dcb-4f27-9da5-76b6974f8566 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:31.137898 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 436/875] 10.222.0.45 () {60 vars in 1141 bytes} [Wed Aug 24 17:28:30 2022] POST /identity/v3/services => generated 251 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:28:31.159109 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35087c08-5689-45fd-a277-86444fa782e8 None None] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:31.159442 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35087c08-5689-45fd-a277-86444fa782e8 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:31.159670 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35087c08-5689-45fd-a277-86444fa782e8 None None] PATH_INFO: `/` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:31.161291 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 440/876] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:28:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:28:31.167294 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7defd0-7c24-41c6-a445-34b208e70d24 None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:31.167865 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7defd0-7c24-41c6-a445-34b208e70d24 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:31.168357 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7defd0-7c24-41c6-a445-34b208e70d24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:31.189739 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-ab7defd0-7c24-41c6-a445-34b208e70d24 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:28:31.191357 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-ab7defd0-7c24-41c6-a445-34b208e70d24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:28:31.242478 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35087c08-5689-45fd-a277-86444fa782e8 None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:31.261866 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:31.279862 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 437/877] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:28:31 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 17:28:31.315345 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:31.363153 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35087c08-5689-45fd-a277-86444fa782e8 None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:31.385425 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92b6629-361f-4eca-81a0-47bdd70142bc None None] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:31.386233 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92b6629-361f-4eca-81a0-47bdd70142bc None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:31.386414 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92b6629-361f-4eca-81a0-47bdd70142bc None None] PATH_INFO: `/` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:31.388015 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 438/878] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:28:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 17:28:31.393090 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e43679-2e59-46f4-b718-70ad7beef0ac None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:31.393367 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e43679-2e59-46f4-b718-70ad7beef0ac None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:31.393641 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e43679-2e59-46f4-b718-70ad7beef0ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:31.402757 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe30830-813e-48a6-ab28-3639cb55b9a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:31.403674 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe30830-813e-48a6-ab28-3639cb55b9a1 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:31.404175 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe30830-813e-48a6-ab28-3639cb55b9a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:31.404699 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe30830-813e-48a6-ab28-3639cb55b9a1 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:31.411630 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-a2e43679-2e59-46f4-b718-70ad7beef0ac None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:28:31.412960 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-a2e43679-2e59-46f4-b718-70ad7beef0ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:28:31.416515 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe30830-813e-48a6-ab28-3639cb55b9a1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:28:31.421641 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application [None req-ffe30830-813e-48a6-ab28-3639cb55b9a1 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 17:28:31.421641 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:28:31.421641 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:28:31.421641 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:28:31.421641 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:28:31.421641 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:28:31.421641 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:28:31.421641 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:28:31.421641 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:28:31.421641 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:28:31.421641 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:28:31.421641 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:28:31.421641 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:28:31.421641 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:28:31.421641 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:28:31.421641 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:28:31.422586 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:28:31.422586 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:28:31.422586 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:28:31.422586 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:28:31.422586 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:28:31.422586 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:28:31.422586 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:28:31.422586 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:28:31.422586 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:28:31.422586 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:28:31.422586 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:28:31.422586 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application with Lock( Aug 24 17:28:31.422586 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:28:31.422586 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:28:31.422586 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:28:31.422586 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:28:31.423432 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:28:31.423432 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:28:31.423432 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:28:31.423432 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:28:31.423432 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:28:31.423432 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:28:31.423432 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:28:31.423432 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:28:31.423432 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:28:31.423432 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:28:31.423432 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 17:28:31.423432 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Aug 24 17:28:31.426177 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 441/879] 10.222.0.45 () {56 vars in 1120 bytes} [Wed Aug 24 17:28:31 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:28:31.432817 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe30830-813e-48a6-ab28-3639cb55b9a1 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:31.496215 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 439/880] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:28:31 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 17:28:31.576173 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e43679-2e59-46f4-b718-70ad7beef0ac None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:31.664093 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe30830-813e-48a6-ab28-3639cb55b9a1 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:31.684063 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:31.702776 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd04f8d-76cb-4e93-8c97-c35f02d952e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:31.703385 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd04f8d-76cb-4e93-8c97-c35f02d952e3 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:31.703664 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd04f8d-76cb-4e93-8c97-c35f02d952e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:31.704050 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd04f8d-76cb-4e93-8c97-c35f02d952e3 None admin] PATH_INFO: `/v3/roles` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:31.704778 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd04f8d-76cb-4e93-8c97-c35f02d952e3 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:31.705067 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd04f8d-76cb-4e93-8c97-c35f02d952e3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:31.706439 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd04f8d-76cb-4e93-8c97-c35f02d952e3 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:31.714090 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e43679-2e59-46f4-b718-70ad7beef0ac None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:31.719077 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 442/881] 10.222.0.45 () {56 vars in 1115 bytes} [Wed Aug 24 17:28:31 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:31.724517 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd04f8d-76cb-4e93-8c97-c35f02d952e3 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:31.742314 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:31.759094 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb21341-db46-43f5-b209-f0559da032b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:31.759708 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb21341-db46-43f5-b209-f0559da032b5 None admin] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:31.759987 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb21341-db46-43f5-b209-f0559da032b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:31.760847 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb21341-db46-43f5-b209-f0559da032b5 None admin] PATH_INFO: `/v3/services` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:31.761556 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb21341-db46-43f5-b209-f0559da032b5 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:31.763023 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb21341-db46-43f5-b209-f0559da032b5 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:31.782826 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 440/882] 10.222.0.45 () {60 vars in 1141 bytes} [Wed Aug 24 17:28:31 2022] POST /identity/v3/services => generated 269 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:28:31.783654 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb21341-db46-43f5-b209-f0559da032b5 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:31.862043 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd04f8d-76cb-4e93-8c97-c35f02d952e3 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:31.905445 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d53dc9-38ad-48ed-958d-cb65b824c400 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:31.906093 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d53dc9-38ad-48ed-958d-cb65b824c400 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:31.906374 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d53dc9-38ad-48ed-958d-cb65b824c400 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:31.906645 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d53dc9-38ad-48ed-958d-cb65b824c400 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:31.918498 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d53dc9-38ad-48ed-958d-cb65b824c400 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:31.919894 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d53dc9-38ad-48ed-958d-cb65b824c400 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:31.921513 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb21341-db46-43f5-b209-f0559da032b5 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:31.939376 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application [None req-24d53dc9-38ad-48ed-958d-cb65b824c400 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:28:31.939376 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:28:31.939376 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:28:31.939376 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:28:31.939376 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:28:31.939376 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:28:31.939376 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:28:31.939376 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:28:31.939376 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:28:31.939376 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:28:31.939376 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:28:31.939376 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:28:31.939376 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:28:31.939376 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:28:31.939376 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:28:31.939376 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:28:31.940397 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:28:31.940397 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:28:31.940397 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:28:31.940397 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:28:31.940397 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:28:31.940397 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:28:31.940397 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:28:31.940397 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application with Lock( Aug 24 17:28:31.940397 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:28:31.940397 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:28:31.940397 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:28:31.940397 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:28:31.940397 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:28:31.940397 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:28:31.940397 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:28:31.940397 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:28:31.940397 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:28:31.941351 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:28:31.941351 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:28:31.941351 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Aug 24 17:28:31.941950 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 443/883] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 17:28:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 17:28:31.948352 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d53dc9-38ad-48ed-958d-cb65b824c400 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:31.969593 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30971a2d-d29f-40a0-b777-ee25ff7a2cb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:31.970235 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30971a2d-d29f-40a0-b777-ee25ff7a2cb3 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:31.970509 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30971a2d-d29f-40a0-b777-ee25ff7a2cb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:31.970778 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30971a2d-d29f-40a0-b777-ee25ff7a2cb3 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:31.971368 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30971a2d-d29f-40a0-b777-ee25ff7a2cb3 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:31.972801 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30971a2d-d29f-40a0-b777-ee25ff7a2cb3 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:31.983135 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application [None req-30971a2d-d29f-40a0-b777-ee25ff7a2cb3 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 17:28:31.983135 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:28:31.983135 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 17:28:31.983135 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 17:28:31.983135 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 17:28:31.983135 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 17:28:31.983135 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 17:28:31.983135 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:28:31.983135 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 17:28:31.983135 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Aug 24 17:28:31.983135 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 17:28:31.983135 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Aug 24 17:28:31.983135 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:28:31.983135 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:28:31.983135 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:28:31.983135 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:28:31.983135 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:28:31.984148 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:28:31.984148 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:28:31.984148 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:28:31.984148 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:28:31.984148 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:28:31.984148 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:28:31.984148 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 17:28:31.984148 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 17:28:31.984148 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 17:28:31.984148 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 17:28:31.984148 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:28:31.984148 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:28:31.984148 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:28:31.984148 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:28:31.984148 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:28:31.984148 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:28:31.985056 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:28:31.985056 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application with Lock( Aug 24 17:28:31.985056 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:28:31.985056 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:28:31.985056 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:28:31.985056 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:28:31.985056 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:28:31.985056 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:28:31.985056 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:28:31.985056 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:28:31.985056 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 17:28:31.985056 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:28:31.985056 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 17:28:31.985056 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Aug 24 17:28:31.985056 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 441/884] 10.222.0.45 () {56 vars in 1126 bytes} [Wed Aug 24 17:28:31 2022] GET /identity/v3/services/object-store => generated 93 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:28:31.990267 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30971a2d-d29f-40a0-b777-ee25ff7a2cb3 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:31.997460 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:32.048945 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:32.080756 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d53dc9-38ad-48ed-958d-cb65b824c400 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:32.116458 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce848c4-f846-4ad7-88d4-a860745c1801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:32.117443 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce848c4-f846-4ad7-88d4-a860745c1801 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:32.117908 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce848c4-f846-4ad7-88d4-a860745c1801 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:32.118374 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce848c4-f846-4ad7-88d4-a860745c1801 None admin] PATH_INFO: `/v3/domains` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:32.119200 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce848c4-f846-4ad7-88d4-a860745c1801 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:32.120233 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce848c4-f846-4ad7-88d4-a860745c1801 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:32.120729 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30971a2d-d29f-40a0-b777-ee25ff7a2cb3 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:32.122067 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce848c4-f846-4ad7-88d4-a860745c1801 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:32.134815 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 444/885] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 24 17:28:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:28:32.140232 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce848c4-f846-4ad7-88d4-a860745c1801 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:32.169889 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a967bb03-c4d3-40df-ab03-64e78fc8d0bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:32.170596 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a967bb03-c4d3-40df-ab03-64e78fc8d0bc None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:32.171190 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a967bb03-c4d3-40df-ab03-64e78fc8d0bc None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:32.171473 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a967bb03-c4d3-40df-ab03-64e78fc8d0bc None admin] PATH_INFO: `/v3/services` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:32.172129 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a967bb03-c4d3-40df-ab03-64e78fc8d0bc None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:32.172434 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a967bb03-c4d3-40df-ab03-64e78fc8d0bc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:32.174595 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a967bb03-c4d3-40df-ab03-64e78fc8d0bc None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:32.180598 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 442/886] 10.222.0.45 () {56 vars in 1122 bytes} [Wed Aug 24 17:28:31 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:32.186319 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a967bb03-c4d3-40df-ab03-64e78fc8d0bc None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:32.264121 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:32.286072 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce848c4-f846-4ad7-88d4-a860745c1801 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:32.315131 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a967bb03-c4d3-40df-ab03-64e78fc8d0bc None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:32.327117 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:32.329195 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663ff061-efda-4430-8ca3-dce39c9b83ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:32.330951 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663ff061-efda-4430-8ca3-dce39c9b83ce None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:32.331381 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663ff061-efda-4430-8ca3-dce39c9b83ce None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:32.331832 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663ff061-efda-4430-8ca3-dce39c9b83ce None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:32.344353 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663ff061-efda-4430-8ca3-dce39c9b83ce None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:32.346063 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663ff061-efda-4430-8ca3-dce39c9b83ce None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:32.356184 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6a3ed5-42aa-41c6-863b-75c555228976 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:32.356839 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6a3ed5-42aa-41c6-863b-75c555228976 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:32.357125 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6a3ed5-42aa-41c6-863b-75c555228976 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:32.357405 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6a3ed5-42aa-41c6-863b-75c555228976 None admin] PATH_INFO: `/v3/services` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:32.358044 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6a3ed5-42aa-41c6-863b-75c555228976 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:32.358354 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6a3ed5-42aa-41c6-863b-75c555228976 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:32.359720 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6a3ed5-42aa-41c6-863b-75c555228976 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:32.360706 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application [None req-663ff061-efda-4430-8ca3-dce39c9b83ce None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 17:28:32.360706 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:28:32.360706 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:28:32.360706 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:28:32.360706 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:28:32.360706 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:28:32.360706 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:28:32.360706 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:28:32.360706 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:28:32.360706 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:28:32.360706 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:28:32.360706 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:28:32.360706 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 17:28:32.360706 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 17:28:32.360706 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 17:28:32.360706 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 17:28:32.361596 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:28:32.361596 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:28:32.361596 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 17:28:32.361596 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:28:32.361596 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 17:28:32.361596 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:28:32.361596 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:28:32.361596 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:28:32.361596 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:28:32.361596 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:28:32.361596 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:28:32.361596 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application with Lock( Aug 24 17:28:32.361596 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:28:32.361596 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:28:32.361596 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:28:32.361596 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:28:32.361596 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:28:32.361596 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:28:32.362514 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:28:32.362514 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:28:32.362514 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 17:28:32.362514 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 17:28:32.362514 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 17:28:32.362514 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 17:28:32.362514 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 17:28:32.362514 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 17:28:32.362514 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 17:28:32.362514 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Aug 24 17:28:32.364698 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 443/887] 10.222.0.45 () {56 vars in 1122 bytes} [Wed Aug 24 17:28:32 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:32.365070 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 445/888] 10.222.0.45 () {56 vars in 1117 bytes} [Wed Aug 24 17:28:32 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 17:28:32.370825 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6a3ed5-42aa-41c6-863b-75c555228976 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:32.485615 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6a3ed5-42aa-41c6-863b-75c555228976 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:32.525635 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a4774f-dd6c-4454-8ec6-40fd8aae7955 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:32.526311 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a4774f-dd6c-4454-8ec6-40fd8aae7955 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:32.526592 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a4774f-dd6c-4454-8ec6-40fd8aae7955 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:32.526868 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a4774f-dd6c-4454-8ec6-40fd8aae7955 None admin] PATH_INFO: `/v3/users` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:32.527604 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a4774f-dd6c-4454-8ec6-40fd8aae7955 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:32.527912 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a4774f-dd6c-4454-8ec6-40fd8aae7955 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:32.529364 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a4774f-dd6c-4454-8ec6-40fd8aae7955 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:32.549853 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 444/889] 10.222.0.45 () {56 vars in 1149 bytes} [Wed Aug 24 17:28:32 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:32.555521 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663ff061-efda-4430-8ca3-dce39c9b83ce None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:32.661663 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663ff061-efda-4430-8ca3-dce39c9b83ce None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:32.696990 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff844bc-8fd9-43f6-b89b-7285d62dcda7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:32.697695 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff844bc-8fd9-43f6-b89b-7285d62dcda7 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:32.697939 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff844bc-8fd9-43f6-b89b-7285d62dcda7 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:32.698148 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff844bc-8fd9-43f6-b89b-7285d62dcda7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:32.716997 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:32.768971 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff844bc-8fd9-43f6-b89b-7285d62dcda7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:32.770418 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff844bc-8fd9-43f6-b89b-7285d62dcda7 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:32.872234 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 446/890] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Aug 24 17:28:32 2022] GET /identity/v3/auth/tokens => generated 1372 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 17:28:32.877688 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a4774f-dd6c-4454-8ec6-40fd8aae7955 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:32.911696 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:32.991210 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a4774f-dd6c-4454-8ec6-40fd8aae7955 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:33.027852 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae9e6da-20c1-4ed0-8fe2-12411bf51869 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:33.028403 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae9e6da-20c1-4ed0-8fe2-12411bf51869 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:33.028717 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae9e6da-20c1-4ed0-8fe2-12411bf51869 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:33.028987 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae9e6da-20c1-4ed0-8fe2-12411bf51869 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:33.039739 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae9e6da-20c1-4ed0-8fe2-12411bf51869 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:33.041213 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae9e6da-20c1-4ed0-8fe2-12411bf51869 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:33.055785 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application [None req-3ae9e6da-20c1-4ed0-8fe2-12411bf51869 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:28:33.055785 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:28:33.055785 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:28:33.055785 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:28:33.055785 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:28:33.055785 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:28:33.055785 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:28:33.055785 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:28:33.055785 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:28:33.055785 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:28:33.055785 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:28:33.055785 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:28:33.055785 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:28:33.055785 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:28:33.055785 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:28:33.055785 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:28:33.056782 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:28:33.056782 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:28:33.056782 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:28:33.056782 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:28:33.056782 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:28:33.056782 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:28:33.056782 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:28:33.056782 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application with Lock( Aug 24 17:28:33.056782 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:28:33.056782 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:28:33.056782 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:28:33.056782 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:28:33.056782 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:28:33.056782 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:28:33.056782 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:28:33.056782 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:28:33.056782 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:28:33.057738 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:28:33.057738 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:28:33.057738 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Aug 24 17:28:33.057738 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 445/891] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 17:28:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:28:33.065784 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff844bc-8fd9-43f6-b89b-7285d62dcda7 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:33.119473 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:33.175286 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430afcd1-eaae-4429-9198-970f9b4c5c7d None None] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:33.175480 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff844bc-8fd9-43f6-b89b-7285d62dcda7 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:33.176029 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430afcd1-eaae-4429-9198-970f9b4c5c7d None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:33.176510 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430afcd1-eaae-4429-9198-970f9b4c5c7d None None] PATH_INFO: `/` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:33.178112 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 446/892] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:28:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:28:33.186007 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19918168-55d6-42d8-9afc-3b28216bd4d4 None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:33.186290 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19918168-55d6-42d8-9afc-3b28216bd4d4 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:33.186568 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19918168-55d6-42d8-9afc-3b28216bd4d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:33.211120 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-19918168-55d6-42d8-9afc-3b28216bd4d4 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:28:33.212801 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-19918168-55d6-42d8-9afc-3b28216bd4d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:28:33.215492 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb5d595-9eb6-4764-9531-29f5bae30911 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:33.216111 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb5d595-9eb6-4764-9531-29f5bae30911 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:33.216390 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb5d595-9eb6-4764-9531-29f5bae30911 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:33.216695 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb5d595-9eb6-4764-9531-29f5bae30911 None admin] PATH_INFO: `/v3/domains` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:33.217342 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb5d595-9eb6-4764-9531-29f5bae30911 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:33.217649 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb5d595-9eb6-4764-9531-29f5bae30911 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:33.219017 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb5d595-9eb6-4764-9531-29f5bae30911 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:33.232707 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 447/893] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 24 17:28:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:33.237359 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb5d595-9eb6-4764-9531-29f5bae30911 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:33.301022 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 447/894] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:28:33 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:28:33.354417 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb5d595-9eb6-4764-9531-29f5bae30911 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:33.364319 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:33.389599 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d60ea9-fa55-4b70-9e02-40d7a4a4acf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:33.390236 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d60ea9-fa55-4b70-9e02-40d7a4a4acf4 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:33.390516 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d60ea9-fa55-4b70-9e02-40d7a4a4acf4 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:33.390786 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d60ea9-fa55-4b70-9e02-40d7a4a4acf4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:33.397701 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d60ea9-fa55-4b70-9e02-40d7a4a4acf4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:33.399161 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d60ea9-fa55-4b70-9e02-40d7a4a4acf4 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:33.411101 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application [None req-87d60ea9-fa55-4b70-9e02-40d7a4a4acf4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:28:33.411101 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:28:33.411101 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:28:33.411101 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:28:33.411101 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:28:33.411101 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:28:33.411101 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:28:33.411101 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:28:33.411101 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:28:33.411101 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:28:33.411101 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:28:33.411101 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:28:33.411101 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 17:28:33.411101 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 17:28:33.411101 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 17:28:33.412014 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 17:28:33.412014 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:28:33.412014 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:28:33.412014 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:28:33.412014 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:28:33.412014 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:28:33.412014 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:28:33.412014 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:28:33.412014 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application with Lock( Aug 24 17:28:33.412014 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:28:33.412014 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:28:33.412014 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:28:33.412014 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:28:33.412014 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:28:33.412014 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:28:33.412014 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:28:33.412014 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:28:33.412913 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 17:28:33.412913 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 17:28:33.412913 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 17:28:33.412913 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 17:28:33.412913 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 17:28:33.412913 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 17:28:33.412913 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:28:33.412913 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Aug 24 17:28:33.412913 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 448/895] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Aug 24 17:28:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 17:28:33.419930 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19918168-55d6-42d8-9afc-3b28216bd4d4 None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:33.436171 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:33.527313 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19918168-55d6-42d8-9afc-3b28216bd4d4 None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:33.562483 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37cb867-7a3b-4050-bf26-fa6d1fd62c18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:33.563025 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37cb867-7a3b-4050-bf26-fa6d1fd62c18 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:33.563307 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37cb867-7a3b-4050-bf26-fa6d1fd62c18 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:33.563582 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37cb867-7a3b-4050-bf26-fa6d1fd62c18 None admin] PATH_INFO: `/v3/projects` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:33.564278 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37cb867-7a3b-4050-bf26-fa6d1fd62c18 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:33.564604 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37cb867-7a3b-4050-bf26-fa6d1fd62c18 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:33.566042 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37cb867-7a3b-4050-bf26-fa6d1fd62c18 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:33.578903 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 448/896] 10.222.0.45 () {56 vars in 1148 bytes} [Wed Aug 24 17:28:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:33.585633 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d60ea9-fa55-4b70-9e02-40d7a4a4acf4 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:33.605329 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37cb867-7a3b-4050-bf26-fa6d1fd62c18 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:33.679469 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:33.716477 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37cb867-7a3b-4050-bf26-fa6d1fd62c18 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:33.725361 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d60ea9-fa55-4b70-9e02-40d7a4a4acf4 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:33.764660 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d8d893-0b9c-4006-b85e-fe5e1910abbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:33.765330 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d8d893-0b9c-4006-b85e-fe5e1910abbe None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:33.765617 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d8d893-0b9c-4006-b85e-fe5e1910abbe None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:33.765896 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d8d893-0b9c-4006-b85e-fe5e1910abbe None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:33.766768 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d8d893-0b9c-4006-b85e-fe5e1910abbe None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:33.768275 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d8d893-0b9c-4006-b85e-fe5e1910abbe None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:33.774586 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591e65d9-4f47-411e-9202-3574c4d13d00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:33.776476 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591e65d9-4f47-411e-9202-3574c4d13d00 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:33.777275 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591e65d9-4f47-411e-9202-3574c4d13d00 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:33.777740 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591e65d9-4f47-411e-9202-3574c4d13d00 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:33.778792 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591e65d9-4f47-411e-9202-3574c4d13d00 None admin] RBAC: Adding query filter params (role.id=d6d7b68e2dee4176bb8712ffbac4808f, scope.project.id=2ad8a32bd9d2434097bd08b111914ca1, user.id=ea143d042ee94ae7a4544d90d18db7db) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:33.779283 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591e65d9-4f47-411e-9202-3574c4d13d00 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:33.781031 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591e65d9-4f47-411e-9202-3574c4d13d00 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:33.782842 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application [None req-46d8d893-0b9c-4006-b85e-fe5e1910abbe None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 17:28:33.782842 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:28:33.782842 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 17:28:33.782842 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 17:28:33.782842 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 17:28:33.782842 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 17:28:33.782842 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 17:28:33.782842 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:28:33.782842 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 17:28:33.782842 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Aug 24 17:28:33.782842 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 17:28:33.782842 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Aug 24 17:28:33.782842 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:28:33.782842 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:28:33.782842 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:28:33.782842 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:28:33.782842 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:28:33.783812 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:28:33.783812 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:28:33.783812 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:28:33.783812 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:28:33.783812 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:28:33.783812 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:28:33.783812 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 17:28:33.783812 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 17:28:33.783812 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 17:28:33.783812 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 17:28:33.783812 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:28:33.783812 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:28:33.783812 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:28:33.783812 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:28:33.783812 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:28:33.783812 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:28:33.784667 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:28:33.784667 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application with Lock( Aug 24 17:28:33.784667 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:28:33.784667 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:28:33.784667 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:28:33.784667 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:28:33.784667 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:28:33.784667 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:28:33.784667 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:28:33.784667 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:28:33.784667 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 17:28:33.784667 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:28:33.784667 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 17:28:33.784667 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Aug 24 17:28:33.788629 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 449/897] 10.222.0.45 () {56 vars in 1114 bytes} [Wed Aug 24 17:28:33 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 17:28:33.789927 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d8d893-0b9c-4006-b85e-fe5e1910abbe None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:33.796203 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 449/898] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Aug 24 17:28:33 2022] GET /identity/v3/role_assignments?user.id=ea143d042ee94ae7a4544d90d18db7db&scope.project.id=2ad8a32bd9d2434097bd08b111914ca1&role.id=d6d7b68e2dee4176bb8712ffbac4808f => generated 259 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:33.912580 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d8d893-0b9c-4006-b85e-fe5e1910abbe None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:33.943400 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e827e41-f1aa-4675-9002-6dc2860c0c3b None None] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:33.943684 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e827e41-f1aa-4675-9002-6dc2860c0c3b None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:33.943950 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e827e41-f1aa-4675-9002-6dc2860c0c3b None None] PATH_INFO: `/` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:33.945445 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 450/899] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:28:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 17:28:33.951490 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc36f85-dfb4-49ff-930c-2196c7c6303f None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:33.951772 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc36f85-dfb4-49ff-930c-2196c7c6303f None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:33.952032 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7250be7b-a6c9-4667-aa88-b983becdea3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:33.952155 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc36f85-dfb4-49ff-930c-2196c7c6303f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:33.952669 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7250be7b-a6c9-4667-aa88-b983becdea3c None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:33.952955 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7250be7b-a6c9-4667-aa88-b983becdea3c None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:33.953227 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7250be7b-a6c9-4667-aa88-b983becdea3c None admin] PATH_INFO: `/v3/services` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:33.953863 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7250be7b-a6c9-4667-aa88-b983becdea3c None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:33.954163 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7250be7b-a6c9-4667-aa88-b983becdea3c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:33.955536 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7250be7b-a6c9-4667-aa88-b983becdea3c None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:33.960669 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 450/900] 10.222.0.45 () {56 vars in 1114 bytes} [Wed Aug 24 17:28:33 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:28:33.961235 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:33.970365 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7250be7b-a6c9-4667-aa88-b983becdea3c None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:33.972014 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-1cc36f85-dfb4-49ff-930c-2196c7c6303f None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:28:33.973250 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-1cc36f85-dfb4-49ff-930c-2196c7c6303f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:28:34.009491 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:34.053534 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 451/901] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:28:33 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 17:28:34.094769 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7250be7b-a6c9-4667-aa88-b983becdea3c None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:34.133725 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd4316c-e4e9-44ca-bc92-3413c5a0f218 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:34.134343 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd4316c-e4e9-44ca-bc92-3413c5a0f218 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:34.134623 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd4316c-e4e9-44ca-bc92-3413c5a0f218 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:34.134896 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd4316c-e4e9-44ca-bc92-3413c5a0f218 None admin] PATH_INFO: `/v3/services` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:34.135684 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd4316c-e4e9-44ca-bc92-3413c5a0f218 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:34.135994 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd4316c-e4e9-44ca-bc92-3413c5a0f218 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:34.137434 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd4316c-e4e9-44ca-bc92-3413c5a0f218 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:34.142945 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 451/902] 10.222.0.45 () {56 vars in 1114 bytes} [Wed Aug 24 17:28:33 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:34.148940 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc36f85-dfb4-49ff-930c-2196c7c6303f None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:34.254206 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc36f85-dfb4-49ff-930c-2196c7c6303f None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:34.289153 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cce128-6684-49b6-bc84-5e30a09cbd42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:34.289810 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cce128-6684-49b6-bc84-5e30a09cbd42 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:34.290087 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cce128-6684-49b6-bc84-5e30a09cbd42 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:34.290356 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cce128-6684-49b6-bc84-5e30a09cbd42 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:34.291057 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cce128-6684-49b6-bc84-5e30a09cbd42 None admin] RBAC: Adding query filter params (interface=public, service_id=cfcb8c4b0a6b46ae9222d532eed9dfaa, region_id=RegionOne) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:34.291382 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cce128-6684-49b6-bc84-5e30a09cbd42 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:34.292783 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cce128-6684-49b6-bc84-5e30a09cbd42 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:34.298387 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 452/903] 10.222.0.45 () {56 vars in 1251 bytes} [Wed Aug 24 17:28:34 2022] GET /identity/v3/endpoints?service_id=cfcb8c4b0a6b46ae9222d532eed9dfaa&interface=public®ion_id=RegionOne => generated 194 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:34.303646 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd4316c-e4e9-44ca-bc92-3413c5a0f218 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:34.353633 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cce128-6684-49b6-bc84-5e30a09cbd42 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:34.371181 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:34.384115 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:34.413397 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd4316c-e4e9-44ca-bc92-3413c5a0f218 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:34.449746 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6eabed-b2f2-43c2-858a-6f968244d464 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:34.450363 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6eabed-b2f2-43c2-858a-6f968244d464 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:34.450646 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6eabed-b2f2-43c2-858a-6f968244d464 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:34.451363 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6eabed-b2f2-43c2-858a-6f968244d464 None admin] PATH_INFO: `/v3/services` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:34.451995 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6eabed-b2f2-43c2-858a-6f968244d464 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:34.453403 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6eabed-b2f2-43c2-858a-6f968244d464 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:34.466254 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 452/904] 10.222.0.45 () {56 vars in 1087 bytes} [Wed Aug 24 17:28:34 2022] GET /identity/v3/services => generated 1268 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:28:34.466254 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e7e291-6a55-4308-825f-b15cef9320f5 None None] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:34.466254 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e7e291-6a55-4308-825f-b15cef9320f5 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:34.466254 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e7e291-6a55-4308-825f-b15cef9320f5 None None] PATH_INFO: `/` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:34.466653 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 453/905] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:28:34 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:28:34.468870 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f05b25-8f3f-409b-aafa-7c78fda16fad None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:34.469222 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f05b25-8f3f-409b-aafa-7c78fda16fad None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:34.469627 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f05b25-8f3f-409b-aafa-7c78fda16fad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:34.472823 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cce128-6684-49b6-bc84-5e30a09cbd42 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:34.488912 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-b3f05b25-8f3f-409b-aafa-7c78fda16fad None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:28:34.490221 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-b3f05b25-8f3f-409b-aafa-7c78fda16fad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:28:34.516107 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d5ca84-c6c6-49a4-a194-05bf480fea9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:34.516815 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d5ca84-c6c6-49a4-a194-05bf480fea9d None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:34.517211 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d5ca84-c6c6-49a4-a194-05bf480fea9d None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:34.517551 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d5ca84-c6c6-49a4-a194-05bf480fea9d None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:34.518264 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d5ca84-c6c6-49a4-a194-05bf480fea9d None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:34.519771 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d5ca84-c6c6-49a4-a194-05bf480fea9d None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:34.529624 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application [None req-c0d5ca84-c6c6-49a4-a194-05bf480fea9d None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 17:28:34.529624 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:28:34.529624 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 17:28:34.529624 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 17:28:34.529624 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 17:28:34.529624 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 17:28:34.529624 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 17:28:34.529624 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:28:34.529624 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 17:28:34.529624 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Aug 24 17:28:34.529624 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 17:28:34.529624 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Aug 24 17:28:34.529624 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:28:34.529624 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:28:34.529624 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:28:34.529624 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:28:34.530651 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:28:34.530651 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:28:34.530651 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:28:34.530651 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:28:34.530651 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:28:34.530651 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:28:34.530651 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:28:34.530651 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 17:28:34.530651 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 17:28:34.530651 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 17:28:34.530651 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 17:28:34.530651 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:28:34.530651 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:28:34.530651 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:28:34.530651 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:28:34.530651 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:28:34.530651 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:28:34.531575 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:28:34.531575 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application with Lock( Aug 24 17:28:34.531575 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:28:34.531575 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:28:34.531575 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:28:34.531575 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:28:34.531575 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:28:34.531575 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:28:34.531575 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:28:34.531575 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:28:34.531575 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 17:28:34.531575 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:28:34.531575 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 17:28:34.531575 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Aug 24 17:28:34.531575 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 453/906] 10.222.0.45 () {56 vars in 1132 bytes} [Wed Aug 24 17:28:34 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:28:34.537304 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d5ca84-c6c6-49a4-a194-05bf480fea9d None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:34.586858 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 454/907] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:28:34 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 17:28:34.649388 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:34.659152 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d5ca84-c6c6-49a4-a194-05bf480fea9d None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:34.694516 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5e4891-3558-4dc7-b456-4de06e44727f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:34.695127 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5e4891-3558-4dc7-b456-4de06e44727f None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:34.695400 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5e4891-3558-4dc7-b456-4de06e44727f None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:34.695670 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5e4891-3558-4dc7-b456-4de06e44727f None admin] PATH_INFO: `/v3/services` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:34.696301 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5e4891-3558-4dc7-b456-4de06e44727f None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:34.696698 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5e4891-3558-4dc7-b456-4de06e44727f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:34.698089 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5e4891-3558-4dc7-b456-4de06e44727f None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:34.703091 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 454/908] 10.222.0.45 () {56 vars in 1126 bytes} [Wed Aug 24 17:28:34 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:28:34.708429 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f05b25-8f3f-409b-aafa-7c78fda16fad None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:34.773676 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:34.822117 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f05b25-8f3f-409b-aafa-7c78fda16fad None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:34.858997 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb12a30a-627c-41a4-9f33-0f1078f892ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:34.859406 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb12a30a-627c-41a4-9f33-0f1078f892ef None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:34.859696 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb12a30a-627c-41a4-9f33-0f1078f892ef None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:34.859944 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5e4891-3558-4dc7-b456-4de06e44727f None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:34.860383 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb12a30a-627c-41a4-9f33-0f1078f892ef None admin] PATH_INFO: `/v3/services` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:34.860740 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb12a30a-627c-41a4-9f33-0f1078f892ef None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:34.861047 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb12a30a-627c-41a4-9f33-0f1078f892ef None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:34.862434 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb12a30a-627c-41a4-9f33-0f1078f892ef None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:34.868110 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 455/909] 10.222.0.45 () {56 vars in 1126 bytes} [Wed Aug 24 17:28:34 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:34.876012 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb12a30a-627c-41a4-9f33-0f1078f892ef None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:34.978307 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5e4891-3558-4dc7-b456-4de06e44727f None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:35.011064 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb12a30a-627c-41a4-9f33-0f1078f892ef None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:35.014750 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edc4093-b833-498c-9e82-b37a4c5b8779 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:35.014962 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edc4093-b833-498c-9e82-b37a4c5b8779 None admin] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:35.015238 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edc4093-b833-498c-9e82-b37a4c5b8779 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:35.015508 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edc4093-b833-498c-9e82-b37a4c5b8779 None admin] PATH_INFO: `/v3/services` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:35.016161 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edc4093-b833-498c-9e82-b37a4c5b8779 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:35.017562 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edc4093-b833-498c-9e82-b37a4c5b8779 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:35.040596 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 455/910] 10.222.0.45 () {60 vars in 1141 bytes} [Wed Aug 24 17:28:34 2022] POST /identity/v3/services => generated 241 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:28:35.041359 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:35.060377 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52557ef7-881e-45e9-b71f-8149fb3cb449 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:35.061303 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52557ef7-881e-45e9-b71f-8149fb3cb449 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:35.061735 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52557ef7-881e-45e9-b71f-8149fb3cb449 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:35.062188 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52557ef7-881e-45e9-b71f-8149fb3cb449 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:35.063107 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52557ef7-881e-45e9-b71f-8149fb3cb449 None admin] RBAC: Adding query filter params (interface=public, service_id=50a72a5e534d4b87b42c271e20bd849f, region_id=RegionOne) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:35.063579 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52557ef7-881e-45e9-b71f-8149fb3cb449 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:35.065189 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52557ef7-881e-45e9-b71f-8149fb3cb449 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:35.074916 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 456/911] 10.222.0.45 () {56 vars in 1251 bytes} [Wed Aug 24 17:28:34 2022] GET /identity/v3/endpoints?service_id=50a72a5e534d4b87b42c271e20bd849f&interface=public®ion_id=RegionOne => generated 194 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:28:35.080903 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edc4093-b833-498c-9e82-b37a4c5b8779 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:35.099457 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:35.201717 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edc4093-b833-498c-9e82-b37a4c5b8779 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:35.238334 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179845f3-7b92-4164-b4f4-e8b2d4573606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:35.238942 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179845f3-7b92-4164-b4f4-e8b2d4573606 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:35.239215 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179845f3-7b92-4164-b4f4-e8b2d4573606 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:35.239483 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179845f3-7b92-4164-b4f4-e8b2d4573606 None admin] PATH_INFO: `/v3/services` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:35.240087 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179845f3-7b92-4164-b4f4-e8b2d4573606 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:35.241479 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179845f3-7b92-4164-b4f4-e8b2d4573606 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:35.248690 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 456/912] 10.222.0.45 () {56 vars in 1087 bytes} [Wed Aug 24 17:28:35 2022] GET /identity/v3/services => generated 1497 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:28:35.747754 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d639d587-7182-4bb1-bf41-9f99785b13b9 None None] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:35.748148 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d639d587-7182-4bb1-bf41-9f99785b13b9 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:35.748356 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d639d587-7182-4bb1-bf41-9f99785b13b9 None None] PATH_INFO: `/` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:35.749952 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 457/913] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:28:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:28:35.755263 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4ab5fa-c9f9-490f-b2c9-df70494f5e62 None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:35.755551 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4ab5fa-c9f9-490f-b2c9-df70494f5e62 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:35.755830 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4ab5fa-c9f9-490f-b2c9-df70494f5e62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:35.779427 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-cb4ab5fa-c9f9-490f-b2c9-df70494f5e62 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:28:35.780668 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-cb4ab5fa-c9f9-490f-b2c9-df70494f5e62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:28:35.856672 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 457/914] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:28:35 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:28:36.127790 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d639d587-7182-4bb1-bf41-9f99785b13b9 None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:36.237236 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d639d587-7182-4bb1-bf41-9f99785b13b9 None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:36.277431 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbf1d05-e6cc-4e12-a7ef-b3746aa175d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:36.278371 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbf1d05-e6cc-4e12-a7ef-b3746aa175d5 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:36.278798 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbf1d05-e6cc-4e12-a7ef-b3746aa175d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:36.279213 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbf1d05-e6cc-4e12-a7ef-b3746aa175d5 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:36.290380 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbf1d05-e6cc-4e12-a7ef-b3746aa175d5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:28:36.295826 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application [None req-1dbf1d05-e6cc-4e12-a7ef-b3746aa175d5 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 17:28:36.295826 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:28:36.295826 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:28:36.295826 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:28:36.295826 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:28:36.295826 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:28:36.295826 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:28:36.295826 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:28:36.295826 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:28:36.295826 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:28:36.295826 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:28:36.295826 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:28:36.295826 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:28:36.295826 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:28:36.295826 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:28:36.295826 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:28:36.296948 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:28:36.296948 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:28:36.296948 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:28:36.296948 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:28:36.296948 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:28:36.296948 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:28:36.296948 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:28:36.296948 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:28:36.296948 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:28:36.296948 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:28:36.296948 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:28:36.296948 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application with Lock( Aug 24 17:28:36.296948 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:28:36.296948 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:28:36.296948 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:28:36.296948 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:28:36.297980 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:28:36.297980 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:28:36.297980 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:28:36.297980 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:28:36.297980 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:28:36.297980 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:28:36.297980 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:28:36.297980 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:28:36.297980 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:28:36.297980 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:28:36.297980 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 17:28:36.297980 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Aug 24 17:28:36.300142 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 458/915] 10.222.0.45 () {56 vars in 1120 bytes} [Wed Aug 24 17:28:36 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:28:36.305905 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4ab5fa-c9f9-490f-b2c9-df70494f5e62 None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:36.317572 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:36.416630 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4ab5fa-c9f9-490f-b2c9-df70494f5e62 None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:36.456126 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca8644f-cb9f-48ec-92e8-db49e8f0380a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:36.457223 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca8644f-cb9f-48ec-92e8-db49e8f0380a None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:36.457705 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca8644f-cb9f-48ec-92e8-db49e8f0380a None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:36.458220 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca8644f-cb9f-48ec-92e8-db49e8f0380a None admin] PATH_INFO: `/v3/roles` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:36.459087 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca8644f-cb9f-48ec-92e8-db49e8f0380a None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:36.459585 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca8644f-cb9f-48ec-92e8-db49e8f0380a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:36.461187 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca8644f-cb9f-48ec-92e8-db49e8f0380a None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:36.470444 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 458/916] 10.222.0.45 () {56 vars in 1115 bytes} [Wed Aug 24 17:28:36 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:36.476076 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7ebf54-b918-42c0-aac2-0ffb756cf8b0 None None] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:36.476355 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7ebf54-b918-42c0-aac2-0ffb756cf8b0 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:36.476657 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7ebf54-b918-42c0-aac2-0ffb756cf8b0 None None] PATH_INFO: `/` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:36.477485 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca8644f-cb9f-48ec-92e8-db49e8f0380a None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:36.478243 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 459/917] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:28:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:28:36.484173 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192da024-c82a-4d41-b970-c03c31f9d83f None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:36.484790 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192da024-c82a-4d41-b970-c03c31f9d83f None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:36.485287 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192da024-c82a-4d41-b970-c03c31f9d83f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:36.503519 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-192da024-c82a-4d41-b970-c03c31f9d83f None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:28:36.505082 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-192da024-c82a-4d41-b970-c03c31f9d83f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:28:36.562443 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:36.563931 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:36.581475 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 460/918] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:28:36 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 17:28:36.598929 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca8644f-cb9f-48ec-92e8-db49e8f0380a None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:36.634599 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112214a5-0dc5-42a8-9188-9fd6b615d94e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:36.635288 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112214a5-0dc5-42a8-9188-9fd6b615d94e None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:36.635568 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112214a5-0dc5-42a8-9188-9fd6b615d94e None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:36.635843 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112214a5-0dc5-42a8-9188-9fd6b615d94e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:36.646898 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112214a5-0dc5-42a8-9188-9fd6b615d94e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:36.648343 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112214a5-0dc5-42a8-9188-9fd6b615d94e None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:36.663303 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application [None req-112214a5-0dc5-42a8-9188-9fd6b615d94e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:28:36.663303 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:28:36.663303 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:28:36.663303 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:28:36.663303 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:28:36.663303 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:28:36.663303 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:28:36.663303 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:28:36.663303 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:28:36.663303 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:28:36.663303 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:28:36.663303 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:28:36.663303 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:28:36.663303 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:28:36.663303 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:28:36.663303 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:28:36.664257 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:28:36.664257 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:28:36.664257 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:28:36.664257 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:28:36.664257 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:28:36.664257 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:28:36.664257 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:28:36.664257 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application with Lock( Aug 24 17:28:36.664257 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:28:36.664257 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:28:36.664257 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:28:36.664257 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:28:36.664257 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:28:36.664257 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:28:36.664257 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:28:36.664257 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:28:36.664257 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:28:36.665189 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:28:36.665189 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:28:36.665189 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Aug 24 17:28:36.665189 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 459/919] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 17:28:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:28:36.672291 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192da024-c82a-4d41-b970-c03c31f9d83f None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:36.780502 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192da024-c82a-4d41-b970-c03c31f9d83f None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:36.820200 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82632f21-ecb6-4785-9751-aa823785c780 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:36.820833 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82632f21-ecb6-4785-9751-aa823785c780 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:36.821183 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82632f21-ecb6-4785-9751-aa823785c780 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:36.821456 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82632f21-ecb6-4785-9751-aa823785c780 None admin] PATH_INFO: `/v3/domains` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:36.822104 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82632f21-ecb6-4785-9751-aa823785c780 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:36.822399 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82632f21-ecb6-4785-9751-aa823785c780 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:36.823800 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82632f21-ecb6-4785-9751-aa823785c780 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:36.836109 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 461/920] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 24 17:28:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:36.841528 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112214a5-0dc5-42a8-9188-9fd6b615d94e None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:36.860580 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82632f21-ecb6-4785-9751-aa823785c780 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:36.892757 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:36.955574 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112214a5-0dc5-42a8-9188-9fd6b615d94e None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:36.963731 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82632f21-ecb6-4785-9751-aa823785c780 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:36.981392 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:36.996595 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c699f62-fbaa-404b-95ff-3b69514d2293 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:36.997799 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c699f62-fbaa-404b-95ff-3b69514d2293 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:36.998287 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c699f62-fbaa-404b-95ff-3b69514d2293 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:36.998706 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558730ba-f1ea-4c95-b195-7622d079463b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:36.998990 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c699f62-fbaa-404b-95ff-3b69514d2293 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:36.999336 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558730ba-f1ea-4c95-b195-7622d079463b None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:36.999610 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558730ba-f1ea-4c95-b195-7622d079463b None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:36.999878 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558730ba-f1ea-4c95-b195-7622d079463b None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:37.000465 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558730ba-f1ea-4c95-b195-7622d079463b None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:37.001867 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558730ba-f1ea-4c95-b195-7622d079463b None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:37.008755 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c699f62-fbaa-404b-95ff-3b69514d2293 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:37.010530 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c699f62-fbaa-404b-95ff-3b69514d2293 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:37.010908 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application [None req-558730ba-f1ea-4c95-b195-7622d079463b None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 17:28:37.010908 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:28:37.010908 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 17:28:37.010908 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 17:28:37.010908 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 17:28:37.010908 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 17:28:37.010908 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 17:28:37.010908 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:28:37.010908 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 17:28:37.010908 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Aug 24 17:28:37.010908 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 17:28:37.010908 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Aug 24 17:28:37.010908 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:28:37.010908 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:28:37.010908 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:28:37.010908 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:28:37.010908 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:28:37.011924 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:28:37.011924 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:28:37.011924 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:28:37.011924 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:28:37.011924 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:28:37.011924 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:28:37.011924 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 17:28:37.011924 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 17:28:37.011924 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 17:28:37.011924 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 17:28:37.011924 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:28:37.011924 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:28:37.011924 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:28:37.011924 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:28:37.011924 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:28:37.011924 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:28:37.012788 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:28:37.012788 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application with Lock( Aug 24 17:28:37.012788 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:28:37.012788 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:28:37.012788 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:28:37.012788 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:28:37.012788 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:28:37.012788 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:28:37.012788 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:28:37.012788 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:28:37.012788 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 17:28:37.012788 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:28:37.012788 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 17:28:37.012788 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Aug 24 17:28:37.017167 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 462/921] 10.222.0.45 () {56 vars in 1114 bytes} [Wed Aug 24 17:28:36 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 17:28:37.018186 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558730ba-f1ea-4c95-b195-7622d079463b None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:37.026518 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application [None req-8c699f62-fbaa-404b-95ff-3b69514d2293 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 17:28:37.026518 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:28:37.026518 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:28:37.026518 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:28:37.026518 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:28:37.026518 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:28:37.026518 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:28:37.026518 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:28:37.026518 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:28:37.026518 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:28:37.026518 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:28:37.026518 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:28:37.026518 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 17:28:37.026518 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 17:28:37.026518 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 17:28:37.026518 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 17:28:37.027532 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:28:37.027532 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:28:37.027532 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 17:28:37.027532 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:28:37.027532 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 17:28:37.027532 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:28:37.027532 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:28:37.027532 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:28:37.027532 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:28:37.027532 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:28:37.027532 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:28:37.027532 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application with Lock( Aug 24 17:28:37.027532 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:28:37.027532 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:28:37.027532 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:28:37.027532 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:28:37.027532 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:28:37.027532 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:28:37.028458 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:28:37.028458 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:28:37.028458 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 17:28:37.028458 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 17:28:37.028458 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 17:28:37.028458 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 17:28:37.028458 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 17:28:37.028458 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 17:28:37.028458 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 17:28:37.028458 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Aug 24 17:28:37.031214 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 460/922] 10.222.0.45 () {56 vars in 1117 bytes} [Wed Aug 24 17:28:36 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:28:37.039301 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c699f62-fbaa-404b-95ff-3b69514d2293 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:37.115874 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:37.123859 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558730ba-f1ea-4c95-b195-7622d079463b None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:37.154255 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c699f62-fbaa-404b-95ff-3b69514d2293 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:37.162410 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a5fa8c-302e-48d7-95b7-896d021695c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:37.163650 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a5fa8c-302e-48d7-95b7-896d021695c2 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:37.164131 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a5fa8c-302e-48d7-95b7-896d021695c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:37.164638 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a5fa8c-302e-48d7-95b7-896d021695c2 None admin] PATH_INFO: `/v3/services` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:37.165457 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a5fa8c-302e-48d7-95b7-896d021695c2 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:37.165943 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a5fa8c-302e-48d7-95b7-896d021695c2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:37.168105 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a5fa8c-302e-48d7-95b7-896d021695c2 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:37.174024 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 463/923] 10.222.0.45 () {56 vars in 1114 bytes} [Wed Aug 24 17:28:37 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:37.175445 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29dd517-782d-4092-a1eb-292a7ec65c22 None None] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:37.175912 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29dd517-782d-4092-a1eb-292a7ec65c22 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:37.176361 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29dd517-782d-4092-a1eb-292a7ec65c22 None None] PATH_INFO: `/` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:37.178076 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 464/924] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:28:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:28:37.182405 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29dd517-782d-4092-a1eb-292a7ec65c22 None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:37.192016 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3185d022-4d5f-458b-ab20-886813399496 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:37.192640 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3185d022-4d5f-458b-ab20-886813399496 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:37.192916 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3185d022-4d5f-458b-ab20-886813399496 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:37.193183 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3185d022-4d5f-458b-ab20-886813399496 None admin] PATH_INFO: `/v3/users` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:37.193903 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3185d022-4d5f-458b-ab20-886813399496 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:37.194204 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3185d022-4d5f-458b-ab20-886813399496 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:37.195608 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3185d022-4d5f-458b-ab20-886813399496 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:37.212242 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 461/925] 10.222.0.45 () {56 vars in 1149 bytes} [Wed Aug 24 17:28:37 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:37.213929 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27be71b8-2dc9-489e-bdde-ec28f7b5a781 None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:37.214207 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27be71b8-2dc9-489e-bdde-ec28f7b5a781 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:37.214471 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27be71b8-2dc9-489e-bdde-ec28f7b5a781 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:37.231999 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-27be71b8-2dc9-489e-bdde-ec28f7b5a781 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:28:37.233219 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-27be71b8-2dc9-489e-bdde-ec28f7b5a781 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:28:37.265928 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:37.288458 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29dd517-782d-4092-a1eb-292a7ec65c22 None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:37.297766 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 462/926] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:28:37 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:28:37.299013 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e05d73-3af7-4b71-a810-84730a123b26 None None] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:37.299303 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e05d73-3af7-4b71-a810-84730a123b26 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:37.299576 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e05d73-3af7-4b71-a810-84730a123b26 None None] PATH_INFO: `/` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:37.301083 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 463/927] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:28:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:28:37.301534 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e05d73-3af7-4b71-a810-84730a123b26 None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:37.331988 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4378be-b379-4e2c-9056-a07c48d0e625 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:37.332624 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4378be-b379-4e2c-9056-a07c48d0e625 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:37.332914 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4378be-b379-4e2c-9056-a07c48d0e625 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:37.333186 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4378be-b379-4e2c-9056-a07c48d0e625 None admin] PATH_INFO: `/v3/services` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:37.333998 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4378be-b379-4e2c-9056-a07c48d0e625 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:37.334310 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4378be-b379-4e2c-9056-a07c48d0e625 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:37.335689 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4378be-b379-4e2c-9056-a07c48d0e625 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:37.341239 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 465/928] 10.222.0.45 () {56 vars in 1114 bytes} [Wed Aug 24 17:28:37 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:37.346842 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f67cf7c-3832-4a03-84a5-42153273dc22 None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:37.347132 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f67cf7c-3832-4a03-84a5-42153273dc22 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:37.347407 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f67cf7c-3832-4a03-84a5-42153273dc22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:37.365285 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-3f67cf7c-3832-4a03-84a5-42153273dc22 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:28:37.366653 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-3f67cf7c-3832-4a03-84a5-42153273dc22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:28:37.386549 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:37.415558 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e05d73-3af7-4b71-a810-84730a123b26 None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:37.440588 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 466/929] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:28:37 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:28:37.440588 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f67cf7c-3832-4a03-84a5-42153273dc22 None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:37.455761 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8321d2a4-175b-4fac-8bfa-51e15b54c3d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:37.456731 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8321d2a4-175b-4fac-8bfa-51e15b54c3d9 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:37.457266 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8321d2a4-175b-4fac-8bfa-51e15b54c3d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:37.457748 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8321d2a4-175b-4fac-8bfa-51e15b54c3d9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:37.469990 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8321d2a4-175b-4fac-8bfa-51e15b54c3d9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:37.471767 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8321d2a4-175b-4fac-8bfa-51e15b54c3d9 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:37.490227 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application [None req-8321d2a4-175b-4fac-8bfa-51e15b54c3d9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:28:37.490227 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:28:37.490227 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:28:37.490227 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:28:37.490227 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:28:37.490227 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:28:37.490227 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:28:37.490227 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:28:37.490227 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:28:37.490227 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:28:37.490227 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:28:37.490227 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:28:37.490227 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:28:37.490227 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:28:37.490227 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:28:37.490227 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:28:37.491381 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:28:37.491381 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:28:37.491381 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:28:37.491381 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:28:37.491381 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:28:37.491381 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:28:37.491381 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:28:37.491381 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application with Lock( Aug 24 17:28:37.491381 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:28:37.491381 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:28:37.491381 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:28:37.491381 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:28:37.491381 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:28:37.491381 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:28:37.491381 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:28:37.491381 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:28:37.491381 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:28:37.492365 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:28:37.492365 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:28:37.492365 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Aug 24 17:28:37.494018 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 464/930] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 17:28:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:28:37.499679 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8321d2a4-175b-4fac-8bfa-51e15b54c3d9 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:37.527583 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:37.571721 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f67cf7c-3832-4a03-84a5-42153273dc22 None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:37.607757 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4486f50b-08cf-42bc-a0e0-6d64f514c136 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:37.608391 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4486f50b-08cf-42bc-a0e0-6d64f514c136 None admin] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:37.608711 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4486f50b-08cf-42bc-a0e0-6d64f514c136 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:37.608980 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4486f50b-08cf-42bc-a0e0-6d64f514c136 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:37.609635 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4486f50b-08cf-42bc-a0e0-6d64f514c136 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:37.610225 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8321d2a4-175b-4fac-8bfa-51e15b54c3d9 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:37.610993 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4486f50b-08cf-42bc-a0e0-6d64f514c136 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:37.636834 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 467/931] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 17:28:37 2022] POST /identity/v3/endpoints => generated 346 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:28:37.636834 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4486f50b-08cf-42bc-a0e0-6d64f514c136 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:37.650854 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:37.669192 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d910614f-480b-4565-8c79-d57ac70f4fd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:37.669820 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d910614f-480b-4565-8c79-d57ac70f4fd7 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:37.670144 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d910614f-480b-4565-8c79-d57ac70f4fd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:37.670422 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d910614f-480b-4565-8c79-d57ac70f4fd7 None admin] PATH_INFO: `/v3/domains` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:37.671310 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d910614f-480b-4565-8c79-d57ac70f4fd7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:37.671616 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d910614f-480b-4565-8c79-d57ac70f4fd7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:37.673230 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d910614f-480b-4565-8c79-d57ac70f4fd7 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:37.687796 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 465/932] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 24 17:28:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:28:37.693171 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d910614f-480b-4565-8c79-d57ac70f4fd7 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:37.764911 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4486f50b-08cf-42bc-a0e0-6d64f514c136 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:37.784909 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:37.811373 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e803db3-1fde-4fc7-a4b2-e57d90e617d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:37.812018 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e803db3-1fde-4fc7-a4b2-e57d90e617d9 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:37.812305 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e803db3-1fde-4fc7-a4b2-e57d90e617d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:37.812673 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e803db3-1fde-4fc7-a4b2-e57d90e617d9 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:37.813257 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e803db3-1fde-4fc7-a4b2-e57d90e617d9 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:37.814211 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d910614f-480b-4565-8c79-d57ac70f4fd7 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:37.814897 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e803db3-1fde-4fc7-a4b2-e57d90e617d9 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:37.824122 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application [None req-3e803db3-1fde-4fc7-a4b2-e57d90e617d9 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 17:28:37.824122 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:28:37.824122 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 17:28:37.824122 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 17:28:37.824122 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 17:28:37.824122 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 17:28:37.824122 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 17:28:37.824122 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:28:37.824122 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 17:28:37.824122 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Aug 24 17:28:37.824122 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 17:28:37.824122 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Aug 24 17:28:37.824122 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:28:37.824122 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:28:37.824122 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:28:37.824122 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:28:37.824122 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:28:37.825214 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:28:37.825214 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:28:37.825214 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:28:37.825214 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:28:37.825214 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:28:37.825214 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:28:37.825214 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 17:28:37.825214 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 17:28:37.825214 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 17:28:37.825214 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 17:28:37.825214 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:28:37.825214 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:28:37.825214 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:28:37.825214 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:28:37.825214 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:28:37.825214 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:28:37.826041 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:28:37.826041 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application with Lock( Aug 24 17:28:37.826041 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:28:37.826041 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:28:37.826041 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:28:37.826041 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:28:37.826041 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:28:37.826041 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:28:37.826041 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:28:37.826041 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:28:37.826041 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 17:28:37.826041 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:28:37.826041 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 17:28:37.826041 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Aug 24 17:28:37.826041 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 468/933] 10.222.0.45 () {56 vars in 1126 bytes} [Wed Aug 24 17:28:37 2022] GET /identity/v3/services/object-store => generated 93 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:28:37.826842 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e803db3-1fde-4fc7-a4b2-e57d90e617d9 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:37.852747 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e663fdb6-c7f0-414c-bda0-445e2ebadc19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:37.853767 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e663fdb6-c7f0-414c-bda0-445e2ebadc19 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:37.854325 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e663fdb6-c7f0-414c-bda0-445e2ebadc19 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:37.854795 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e663fdb6-c7f0-414c-bda0-445e2ebadc19 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:37.862331 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e663fdb6-c7f0-414c-bda0-445e2ebadc19 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:37.863988 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e663fdb6-c7f0-414c-bda0-445e2ebadc19 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:37.876425 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application [None req-e663fdb6-c7f0-414c-bda0-445e2ebadc19 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:28:37.876425 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:28:37.876425 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:28:37.876425 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:28:37.876425 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:28:37.876425 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:28:37.876425 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:28:37.876425 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:28:37.876425 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:28:37.876425 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:28:37.876425 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:28:37.876425 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:28:37.876425 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 17:28:37.876425 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 17:28:37.876425 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 17:28:37.877609 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 17:28:37.877609 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:28:37.877609 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:28:37.877609 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:28:37.877609 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:28:37.877609 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:28:37.877609 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:28:37.877609 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:28:37.877609 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application with Lock( Aug 24 17:28:37.877609 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:28:37.877609 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:28:37.877609 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:28:37.877609 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:28:37.877609 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:28:37.877609 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:28:37.877609 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:28:37.877609 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:28:37.878730 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 17:28:37.878730 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 17:28:37.878730 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 17:28:37.878730 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 17:28:37.878730 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 17:28:37.878730 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 17:28:37.878730 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:28:37.878730 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Aug 24 17:28:37.880780 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 466/934] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Aug 24 17:28:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 17:28:37.881719 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e663fdb6-c7f0-414c-bda0-445e2ebadc19 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:37.911318 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:37.938561 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e803db3-1fde-4fc7-a4b2-e57d90e617d9 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:37.975904 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e1b5e3-a8ab-4e8e-a6d4-8d6c89108584 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:37.976573 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e1b5e3-a8ab-4e8e-a6d4-8d6c89108584 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:37.976859 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e1b5e3-a8ab-4e8e-a6d4-8d6c89108584 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:37.977135 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e1b5e3-a8ab-4e8e-a6d4-8d6c89108584 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:37.977731 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e1b5e3-a8ab-4e8e-a6d4-8d6c89108584 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:37.979130 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e1b5e3-a8ab-4e8e-a6d4-8d6c89108584 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:37.988443 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application [None req-16e1b5e3-a8ab-4e8e-a6d4-8d6c89108584 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 17:28:37.988443 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:28:37.988443 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 17:28:37.988443 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 17:28:37.988443 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 17:28:37.988443 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 17:28:37.988443 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 17:28:37.988443 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:28:37.988443 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 17:28:37.988443 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Aug 24 17:28:37.988443 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 17:28:37.988443 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Aug 24 17:28:37.988443 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:28:37.988443 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:28:37.988443 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:28:37.988443 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:28:37.989618 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:28:37.989618 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:28:37.989618 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:28:37.989618 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:28:37.989618 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:28:37.989618 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:28:37.989618 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:28:37.989618 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 17:28:37.989618 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 17:28:37.989618 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 17:28:37.989618 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 17:28:37.989618 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:28:37.989618 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:28:37.989618 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:28:37.989618 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:28:37.989618 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:28:37.989618 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:28:37.990539 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:28:37.990539 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application with Lock( Aug 24 17:28:37.990539 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:28:37.990539 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:28:37.990539 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:28:37.990539 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:28:37.990539 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:28:37.990539 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:28:37.990539 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:28:37.990539 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:28:37.990539 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 17:28:37.990539 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:28:37.990539 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 17:28:37.990539 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Aug 24 17:28:37.990539 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 469/935] 10.222.0.45 () {56 vars in 1132 bytes} [Wed Aug 24 17:28:37 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:28:37.991347 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e1b5e3-a8ab-4e8e-a6d4-8d6c89108584 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:38.002218 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e663fdb6-c7f0-414c-bda0-445e2ebadc19 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:38.039973 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd92e64c-71bd-4ede-847f-7917a17981ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:38.040492 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd92e64c-71bd-4ede-847f-7917a17981ac None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:38.040800 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd92e64c-71bd-4ede-847f-7917a17981ac None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:38.041071 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd92e64c-71bd-4ede-847f-7917a17981ac None admin] PATH_INFO: `/v3/services` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:38.041727 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd92e64c-71bd-4ede-847f-7917a17981ac None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:38.042268 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd92e64c-71bd-4ede-847f-7917a17981ac None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:38.043384 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd92e64c-71bd-4ede-847f-7917a17981ac None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:38.048467 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 467/936] 10.222.0.45 () {56 vars in 1122 bytes} [Wed Aug 24 17:28:37 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:38.049316 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd92e64c-71bd-4ede-847f-7917a17981ac None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:38.051978 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:38.114935 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e1b5e3-a8ab-4e8e-a6d4-8d6c89108584 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:38.153202 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820cd5e1-989f-48c6-8656-fbd19aee0da7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:38.153864 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820cd5e1-989f-48c6-8656-fbd19aee0da7 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:38.154150 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820cd5e1-989f-48c6-8656-fbd19aee0da7 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:38.154429 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820cd5e1-989f-48c6-8656-fbd19aee0da7 None admin] PATH_INFO: `/v3/projects` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:38.155121 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820cd5e1-989f-48c6-8656-fbd19aee0da7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:38.155493 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820cd5e1-989f-48c6-8656-fbd19aee0da7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:38.156776 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd92e64c-71bd-4ede-847f-7917a17981ac None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:38.157203 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820cd5e1-989f-48c6-8656-fbd19aee0da7 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:38.168685 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:38.173396 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 470/937] 10.222.0.45 () {56 vars in 1148 bytes} [Wed Aug 24 17:28:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:38.174283 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820cd5e1-989f-48c6-8656-fbd19aee0da7 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:38.198359 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c56c14-e5f7-4d35-9e4e-cb20319f054b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:38.199312 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c56c14-e5f7-4d35-9e4e-cb20319f054b None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:38.199603 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c56c14-e5f7-4d35-9e4e-cb20319f054b None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:38.199873 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c56c14-e5f7-4d35-9e4e-cb20319f054b None admin] PATH_INFO: `/v3/services` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:38.200507 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c56c14-e5f7-4d35-9e4e-cb20319f054b None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:38.200842 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c56c14-e5f7-4d35-9e4e-cb20319f054b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:38.202227 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c56c14-e5f7-4d35-9e4e-cb20319f054b None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:38.207022 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 468/938] 10.222.0.45 () {56 vars in 1126 bytes} [Wed Aug 24 17:28:38 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:38.207786 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c56c14-e5f7-4d35-9e4e-cb20319f054b None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:38.279117 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:38.297603 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820cd5e1-989f-48c6-8656-fbd19aee0da7 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:38.317836 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c56c14-e5f7-4d35-9e4e-cb20319f054b None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:38.336290 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d32c02-d091-45fc-8fcf-9bae2c4d41d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:38.336927 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d32c02-d091-45fc-8fcf-9bae2c4d41d4 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:38.337219 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d32c02-d091-45fc-8fcf-9bae2c4d41d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:38.337520 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d32c02-d091-45fc-8fcf-9bae2c4d41d4 None admin] PATH_INFO: `/v3/services` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:38.338109 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d32c02-d091-45fc-8fcf-9bae2c4d41d4 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:38.338448 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d32c02-d091-45fc-8fcf-9bae2c4d41d4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:38.339805 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d32c02-d091-45fc-8fcf-9bae2c4d41d4 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:38.344790 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 471/939] 10.222.0.45 () {56 vars in 1122 bytes} [Wed Aug 24 17:28:38 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:38.345577 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d32c02-d091-45fc-8fcf-9bae2c4d41d4 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:38.364370 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a05bc56-2318-4854-b907-3b4cddf96a54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:38.365685 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a05bc56-2318-4854-b907-3b4cddf96a54 None admin] REQUEST_METHOD: `PUT` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:38.366140 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a05bc56-2318-4854-b907-3b4cddf96a54 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:38.366566 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a05bc56-2318-4854-b907-3b4cddf96a54 None admin] PATH_INFO: `/v3/projects/2ad8a32bd9d2434097bd08b111914ca1/users/ea143d042ee94ae7a4544d90d18db7db/roles/d6d7b68e2dee4176bb8712ffbac4808f` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:38.370251 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a05bc56-2318-4854-b907-3b4cddf96a54 None admin] RBAC: Authorizing `identity:create_grant(project_id=2ad8a32bd9d2434097bd08b111914ca1, user_id=ea143d042ee94ae7a4544d90d18db7db, role_id=d6d7b68e2dee4176bb8712ffbac4808f)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:38.372875 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a05bc56-2318-4854-b907-3b4cddf96a54 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:38.400597 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 469/940] 10.222.0.45 () {58 vars in 1439 bytes} [Wed Aug 24 17:28:38 2022] PUT /identity/v3/projects/2ad8a32bd9d2434097bd08b111914ca1/users/ea143d042ee94ae7a4544d90d18db7db/roles/d6d7b68e2dee4176bb8712ffbac4808f => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:28:38.400597 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a05bc56-2318-4854-b907-3b4cddf96a54 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:38.420899 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:38.467398 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d32c02-d091-45fc-8fcf-9bae2c4d41d4 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:38.508585 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662039b6-4d67-4f4e-bc73-449839929a44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:38.509249 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662039b6-4d67-4f4e-bc73-449839929a44 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:38.509509 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662039b6-4d67-4f4e-bc73-449839929a44 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:38.509784 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662039b6-4d67-4f4e-bc73-449839929a44 None admin] PATH_INFO: `/v3/services` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:38.510465 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662039b6-4d67-4f4e-bc73-449839929a44 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:38.510768 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662039b6-4d67-4f4e-bc73-449839929a44 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:38.512155 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662039b6-4d67-4f4e-bc73-449839929a44 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:38.522991 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 472/941] 10.222.0.45 () {56 vars in 1126 bytes} [Wed Aug 24 17:28:38 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:38.523950 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662039b6-4d67-4f4e-bc73-449839929a44 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:38.559865 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a05bc56-2318-4854-b907-3b4cddf96a54 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:38.597174 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:38.611556 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc96682d-a4cc-402f-a1a4-7e75ebd193c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:38.612177 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc96682d-a4cc-402f-a1a4-7e75ebd193c0 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:38.612474 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc96682d-a4cc-402f-a1a4-7e75ebd193c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:38.612770 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc96682d-a4cc-402f-a1a4-7e75ebd193c0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:38.613500 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc96682d-a4cc-402f-a1a4-7e75ebd193c0 None admin] RBAC: Adding query filter params (interface=public, service_id=9d6a35e85e7d471d84818bf5f0795424, region_id=RegionOne) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:38.613809 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc96682d-a4cc-402f-a1a4-7e75ebd193c0 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:38.615179 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc96682d-a4cc-402f-a1a4-7e75ebd193c0 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:38.620483 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 470/942] 10.222.0.45 () {56 vars in 1251 bytes} [Wed Aug 24 17:28:38 2022] GET /identity/v3/endpoints?service_id=9d6a35e85e7d471d84818bf5f0795424&interface=public®ion_id=RegionOne => generated 194 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:38.626936 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc96682d-a4cc-402f-a1a4-7e75ebd193c0 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:38.664128 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662039b6-4d67-4f4e-bc73-449839929a44 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:38.698871 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:38.711011 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f18626-b846-485c-91e5-e540f3d1ac90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:38.712343 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f18626-b846-485c-91e5-e540f3d1ac90 None admin] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:38.712822 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f18626-b846-485c-91e5-e540f3d1ac90 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:38.713767 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f18626-b846-485c-91e5-e540f3d1ac90 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:38.714995 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f18626-b846-485c-91e5-e540f3d1ac90 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:38.716869 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f18626-b846-485c-91e5-e540f3d1ac90 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:38.738124 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc96682d-a4cc-402f-a1a4-7e75ebd193c0 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:38.746673 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 473/943] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 17:28:38 2022] POST /identity/v3/endpoints => generated 347 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:28:38.796159 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36b8e44-0c9a-472e-bf96-92258686b5bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:38.796813 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36b8e44-0c9a-472e-bf96-92258686b5bf None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:38.797129 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36b8e44-0c9a-472e-bf96-92258686b5bf None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:38.797471 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36b8e44-0c9a-472e-bf96-92258686b5bf None admin] PATH_INFO: `/v3/services` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:38.798142 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36b8e44-0c9a-472e-bf96-92258686b5bf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:38.799526 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36b8e44-0c9a-472e-bf96-92258686b5bf None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:38.806574 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 471/944] 10.222.0.45 () {56 vars in 1087 bytes} [Wed Aug 24 17:28:38 2022] GET /identity/v3/services => generated 1497 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:28:39.633187 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d49f7d9-a124-4881-8ddf-17b37d896469 None None] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:39.633712 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d49f7d9-a124-4881-8ddf-17b37d896469 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:39.633712 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d49f7d9-a124-4881-8ddf-17b37d896469 None None] PATH_INFO: `/` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:39.635817 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 474/945] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:28:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:28:39.641388 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71141fd0-63b3-40af-a19c-f481a48a2fd1 None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:39.641809 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71141fd0-63b3-40af-a19c-f481a48a2fd1 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:39.641984 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71141fd0-63b3-40af-a19c-f481a48a2fd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:39.660709 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-71141fd0-63b3-40af-a19c-f481a48a2fd1 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:28:39.661909 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-71141fd0-63b3-40af-a19c-f481a48a2fd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:28:39.709878 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:39.732787 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 472/946] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:28:39 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 17:28:40.011533 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d49f7d9-a124-4881-8ddf-17b37d896469 None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:40.115385 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d49f7d9-a124-4881-8ddf-17b37d896469 None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:40.150613 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b59c8bd-1cc3-4d69-9ffb-1c08e1d0344b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:40.151789 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b59c8bd-1cc3-4d69-9ffb-1c08e1d0344b None admin] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:40.152067 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b59c8bd-1cc3-4d69-9ffb-1c08e1d0344b None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:40.152341 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b59c8bd-1cc3-4d69-9ffb-1c08e1d0344b None admin] PATH_INFO: `/v3/projects` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:40.153031 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b59c8bd-1cc3-4d69-9ffb-1c08e1d0344b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:40.154473 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b59c8bd-1cc3-4d69-9ffb-1c08e1d0344b None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:40.157133 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: WARNING keystone.server.flask.common [None req-5b59c8bd-1cc3-4d69-9ffb-1c08e1d0344b None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 24 17:28:40.176593 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.sql.core [None req-5b59c8bd-1cc3-4d69-9ffb-1c08e1d0344b None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 24 17:28:40.176593 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 24 17:28:40.176593 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [parameters: {'id': 'b3eaa00bae5c4c66a0afcb09b276e0b2', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 24 17:28:40.176593 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81758) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 24 17:28:40.184072 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application [None req-5b59c8bd-1cc3-4d69-9ffb-1c08e1d0344b None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id b3eaa00bae5c4c66a0afcb09b276e0b2.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id b3eaa00bae5c4c66a0afcb09b276e0b2. Aug 24 17:28:40.184072 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:28:40.184072 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 17:28:40.184072 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 17:28:40.184072 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 17:28:40.184072 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 17:28:40.184072 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 17:28:40.184072 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 17:28:40.184072 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 17:28:40.184072 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application conn.query(q) Aug 24 17:28:40.184072 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 17:28:40.184072 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 17:28:40.184072 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 17:28:40.184925 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application result.read() Aug 24 17:28:40.184925 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 17:28:40.184925 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 17:28:40.184925 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 17:28:40.184925 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 17:28:40.184925 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 17:28:40.184925 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 17:28:40.184925 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 17:28:40.184925 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 17:28:40.184925 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 24 17:28:40.184925 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Aug 24 17:28:40.184925 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 24 17:28:40.184925 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Aug 24 17:28:40.184925 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:28:40.184925 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 24 17:28:40.184925 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 24 17:28:40.184925 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Aug 24 17:28:40.184925 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return project_ref.to_dict() Aug 24 17:28:40.185868 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 17:28:40.185868 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application next(self.gen) Aug 24 17:28:40.185868 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 24 17:28:40.185868 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application yield resource Aug 24 17:28:40.185868 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 17:28:40.185868 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application next(self.gen) Aug 24 17:28:40.185868 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 24 17:28:40.185868 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application self.session.rollback() Aug 24 17:28:40.185868 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 24 17:28:40.185868 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application self.force_reraise() Aug 24 17:28:40.185868 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 24 17:28:40.185868 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application raise self.value Aug 24 17:28:40.185868 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 24 17:28:40.185868 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 24 17:28:40.185868 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 24 17:28:40.185868 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application session.commit() Aug 24 17:28:40.185868 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 24 17:28:40.186758 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 24 17:28:40.186758 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 24 17:28:40.186758 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application self._prepare_impl() Aug 24 17:28:40.186758 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 24 17:28:40.186758 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application self.session.flush() Aug 24 17:28:40.186758 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 24 17:28:40.186758 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application self._flush(objects) Aug 24 17:28:40.186758 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 24 17:28:40.186758 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 24 17:28:40.186758 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 24 17:28:40.186758 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application compat.raise_( Aug 24 17:28:40.186758 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 17:28:40.186758 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application raise exception Aug 24 17:28:40.186758 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 24 17:28:40.186758 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application flush_context.execute() Aug 24 17:28:40.186758 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 24 17:28:40.186758 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application rec.execute(self) Aug 24 17:28:40.187668 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 24 17:28:40.187668 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 24 17:28:40.187668 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 24 17:28:40.187668 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 24 17:28:40.187668 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 24 17:28:40.187668 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 24 17:28:40.187668 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 24 17:28:40.187668 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 24 17:28:40.187668 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 24 17:28:40.187668 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 24 17:28:40.187668 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 24 17:28:40.187668 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 24 17:28:40.187668 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 24 17:28:40.187668 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 24 17:28:40.187668 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 24 17:28:40.188450 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 24 17:28:40.188450 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 17:28:40.188450 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application raise exception Aug 24 17:28:40.188450 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 17:28:40.188450 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 17:28:40.188450 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 17:28:40.188450 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 17:28:40.188450 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 17:28:40.188450 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 17:28:40.188450 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 17:28:40.188450 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application conn.query(q) Aug 24 17:28:40.188450 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 17:28:40.188450 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 17:28:40.188450 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 17:28:40.188450 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application result.read() Aug 24 17:28:40.188450 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 17:28:40.188450 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 17:28:40.191253 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 17:28:40.191253 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 17:28:40.191253 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 17:28:40.191253 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 17:28:40.191253 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 17:28:40.191253 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 17:28:40.191253 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 24 17:28:40.191253 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 24 17:28:40.191253 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application [parameters: {'id': 'b3eaa00bae5c4c66a0afcb09b276e0b2', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 24 17:28:40.191253 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 24 17:28:40.191253 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Aug 24 17:28:40.191253 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 17:28:40.191253 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Aug 24 17:28:40.191253 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:28:40.191253 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Aug 24 17:28:40.193152 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Aug 24 17:28:40.193152 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 24 17:28:40.193152 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 24 17:28:40.193152 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Aug 24 17:28:40.193152 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Aug 24 17:28:40.193152 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 17:28:40.193152 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Aug 24 17:28:40.193152 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:28:40.193152 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:28:40.193152 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:28:40.193152 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:28:40.193152 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:28:40.193152 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:28:40.193152 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:28:40.193152 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:28:40.193152 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:28:40.193152 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:28:40.194505 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:28:40.194505 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Aug 24 17:28:40.194505 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Aug 24 17:28:40.194505 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:28:40.194505 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:28:40.194505 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Aug 24 17:28:40.194505 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application raise exception.Conflict( Aug 24 17:28:40.194505 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id b3eaa00bae5c4c66a0afcb09b276e0b2. Aug 24 17:28:40.194505 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Aug 24 17:28:40.194505 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 475/947] 10.222.0.45 () {60 vars in 1140 bytes} [Wed Aug 24 17:28:40 2022] POST /identity/v3/projects => generated 262 bytes in 175 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:28:40.195160 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71141fd0-63b3-40af-a19c-f481a48a2fd1 None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:40.299997 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71141fd0-63b3-40af-a19c-f481a48a2fd1 None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:40.336507 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e23671f-5302-4acf-8cee-87cefea023fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:40.337221 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e23671f-5302-4acf-8cee-87cefea023fc None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:40.337506 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e23671f-5302-4acf-8cee-87cefea023fc None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:40.337779 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e23671f-5302-4acf-8cee-87cefea023fc None admin] PATH_INFO: `/v3/projects/service` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:40.345585 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e23671f-5302-4acf-8cee-87cefea023fc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:40.347044 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e23671f-5302-4acf-8cee-87cefea023fc None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:40.359349 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application [None req-2e23671f-5302-4acf-8cee-87cefea023fc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:28:40.359349 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:28:40.359349 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:28:40.359349 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:28:40.359349 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:28:40.359349 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:28:40.359349 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:28:40.359349 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:28:40.359349 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:28:40.359349 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:28:40.359349 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:28:40.359349 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:28:40.359349 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 17:28:40.359349 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 17:28:40.359349 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 17:28:40.361272 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 17:28:40.361272 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:28:40.361272 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:28:40.361272 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:28:40.361272 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:28:40.361272 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:28:40.361272 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:28:40.361272 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:28:40.361272 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application with Lock( Aug 24 17:28:40.361272 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:28:40.361272 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:28:40.361272 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:28:40.361272 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:28:40.361272 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:28:40.361272 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:28:40.361272 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:28:40.361272 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:28:40.362552 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 17:28:40.362552 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 17:28:40.362552 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 17:28:40.362552 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 17:28:40.362552 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 17:28:40.362552 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 17:28:40.362552 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:28:40.362552 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Aug 24 17:28:40.362552 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 473/948] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Aug 24 17:28:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:28:40.369954 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b59c8bd-1cc3-4d69-9ffb-1c08e1d0344b None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:40.412620 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:40.447435 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6742558e-f27a-4295-ae54-f14d71f94eca None None] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:40.447810 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6742558e-f27a-4295-ae54-f14d71f94eca None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:40.447996 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6742558e-f27a-4295-ae54-f14d71f94eca None None] PATH_INFO: `/` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:40.449673 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 474/949] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:28:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 17:28:40.455112 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f7ec77-514e-4f72-9772-f54dccc4f1d8 None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:40.455392 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f7ec77-514e-4f72-9772-f54dccc4f1d8 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:40.455664 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f7ec77-514e-4f72-9772-f54dccc4f1d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:40.475051 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-79f7ec77-514e-4f72-9772-f54dccc4f1d8 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:28:40.476456 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-79f7ec77-514e-4f72-9772-f54dccc4f1d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:28:40.491401 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b59c8bd-1cc3-4d69-9ffb-1c08e1d0344b None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:40.515924 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:40.539169 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55efaaf5-32d2-44be-8038-673ee3b518ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:40.539806 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55efaaf5-32d2-44be-8038-673ee3b518ae None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:40.540086 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55efaaf5-32d2-44be-8038-673ee3b518ae None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:40.540362 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55efaaf5-32d2-44be-8038-673ee3b518ae None admin] PATH_INFO: `/v3/projects` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:40.541336 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55efaaf5-32d2-44be-8038-673ee3b518ae None admin] RBAC: Adding query filter params (name=service) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:40.542162 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55efaaf5-32d2-44be-8038-673ee3b518ae None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:40.544525 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55efaaf5-32d2-44be-8038-673ee3b518ae None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:40.560139 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 475/950] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:28:40 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:28:40.568611 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 476/951] 10.222.0.45 () {56 vars in 1112 bytes} [Wed Aug 24 17:28:40 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:40.776710 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3beb260-731d-4245-8910-ec42991867ec None None] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:40.776710 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3beb260-731d-4245-8910-ec42991867ec None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:40.776710 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3beb260-731d-4245-8910-ec42991867ec None None] PATH_INFO: `/` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:40.776710 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 476/952] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:28:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:28:40.783115 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff64a3c0-bc3c-47a3-a568-3a09fac7dfc1 None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:40.783639 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff64a3c0-bc3c-47a3-a568-3a09fac7dfc1 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:40.783639 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff64a3c0-bc3c-47a3-a568-3a09fac7dfc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:40.796696 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbb1bee-040d-4622-ae9e-77949a2e53b7 None None] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:40.797019 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbb1bee-040d-4622-ae9e-77949a2e53b7 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:40.797252 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbb1bee-040d-4622-ae9e-77949a2e53b7 None None] PATH_INFO: `/` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:40.799653 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 477/953] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:28:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:28:40.804531 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-ff64a3c0-bc3c-47a3-a568-3a09fac7dfc1 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:28:40.804823 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b17addb-af72-430f-ab03-97c35c1b8bed None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:40.805111 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b17addb-af72-430f-ab03-97c35c1b8bed None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:40.805383 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b17addb-af72-430f-ab03-97c35c1b8bed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:40.806153 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-ff64a3c0-bc3c-47a3-a568-3a09fac7dfc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:28:40.828926 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-4b17addb-af72-430f-ab03-97c35c1b8bed None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:28:40.830200 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-4b17addb-af72-430f-ab03-97c35c1b8bed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:28:40.890109 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 477/954] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:28:40 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:28:40.891005 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff64a3c0-bc3c-47a3-a568-3a09fac7dfc1 None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:40.902206 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 478/955] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:28:40 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:28:40.920940 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:41.003356 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff64a3c0-bc3c-47a3-a568-3a09fac7dfc1 None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:41.038558 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befd568d-26a5-4327-919f-321a0ca0017c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:41.039197 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befd568d-26a5-4327-919f-321a0ca0017c None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:41.039480 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befd568d-26a5-4327-919f-321a0ca0017c None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:41.039744 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befd568d-26a5-4327-919f-321a0ca0017c None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:41.050644 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befd568d-26a5-4327-919f-321a0ca0017c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:28:41.055522 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application [None req-befd568d-26a5-4327-919f-321a0ca0017c None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 17:28:41.055522 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:28:41.055522 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:28:41.055522 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:28:41.055522 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:28:41.055522 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:28:41.055522 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:28:41.055522 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:28:41.055522 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:28:41.055522 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:28:41.055522 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:28:41.055522 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:28:41.055522 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:28:41.055522 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:28:41.055522 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:28:41.055522 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:28:41.056577 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:28:41.056577 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:28:41.056577 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:28:41.056577 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:28:41.056577 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:28:41.056577 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:28:41.056577 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:28:41.056577 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:28:41.056577 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:28:41.056577 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:28:41.056577 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:28:41.056577 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application with Lock( Aug 24 17:28:41.056577 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:28:41.056577 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:28:41.056577 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:28:41.056577 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:28:41.057842 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:28:41.057842 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:28:41.057842 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:28:41.057842 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:28:41.057842 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:28:41.057842 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:28:41.057842 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:28:41.057842 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:28:41.057842 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:28:41.057842 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:28:41.057842 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 17:28:41.057842 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Aug 24 17:28:41.057842 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 478/956] 10.222.0.45 () {56 vars in 1120 bytes} [Wed Aug 24 17:28:40 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:28:41.062916 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b17addb-af72-430f-ab03-97c35c1b8bed None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:41.165982 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befd568d-26a5-4327-919f-321a0ca0017c None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:41.169324 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:41.175590 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b17addb-af72-430f-ab03-97c35c1b8bed None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:41.216617 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1e5dc2-f45b-4ece-994a-e3dab43260ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:41.217775 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1e5dc2-f45b-4ece-994a-e3dab43260ce None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:41.218061 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1e5dc2-f45b-4ece-994a-e3dab43260ce None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:41.218334 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1e5dc2-f45b-4ece-994a-e3dab43260ce None admin] PATH_INFO: `/v3/roles` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:41.219005 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1e5dc2-f45b-4ece-994a-e3dab43260ce None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:41.219762 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1e5dc2-f45b-4ece-994a-e3dab43260ce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:41.221604 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1e5dc2-f45b-4ece-994a-e3dab43260ce None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:41.233720 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 479/957] 10.222.0.45 () {56 vars in 1115 bytes} [Wed Aug 24 17:28:41 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:41.234924 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1e5dc2-f45b-4ece-994a-e3dab43260ce None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:41.272339 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:41.287243 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befd568d-26a5-4327-919f-321a0ca0017c None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:41.324391 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06db4da0-c6cc-4773-9986-adfaba99d9ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:41.325052 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06db4da0-c6cc-4773-9986-adfaba99d9ea None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:41.325334 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06db4da0-c6cc-4773-9986-adfaba99d9ea None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:41.325607 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06db4da0-c6cc-4773-9986-adfaba99d9ea None admin] PATH_INFO: `/v3/services/compute` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:41.326193 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06db4da0-c6cc-4773-9986-adfaba99d9ea None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:41.327553 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06db4da0-c6cc-4773-9986-adfaba99d9ea None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:41.336270 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application [None req-06db4da0-c6cc-4773-9986-adfaba99d9ea None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 17:28:41.336270 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:28:41.336270 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 17:28:41.336270 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 17:28:41.336270 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 17:28:41.336270 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 17:28:41.336270 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 17:28:41.336270 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:28:41.336270 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 17:28:41.336270 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Aug 24 17:28:41.336270 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 17:28:41.336270 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Aug 24 17:28:41.336270 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:28:41.336270 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:28:41.336270 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:28:41.336270 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:28:41.336270 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:28:41.340971 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:28:41.340971 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:28:41.340971 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:28:41.340971 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:28:41.340971 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:28:41.340971 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:28:41.340971 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 17:28:41.340971 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 17:28:41.340971 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 17:28:41.340971 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 17:28:41.340971 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:28:41.340971 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:28:41.340971 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:28:41.340971 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:28:41.340971 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:28:41.340971 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:28:41.341894 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:28:41.341894 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application with Lock( Aug 24 17:28:41.341894 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:28:41.341894 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:28:41.341894 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:28:41.341894 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:28:41.341894 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:28:41.341894 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:28:41.341894 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:28:41.341894 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:28:41.341894 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 17:28:41.341894 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:28:41.341894 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 17:28:41.341894 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Aug 24 17:28:41.341894 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 479/958] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Aug 24 17:28:41 2022] GET /identity/v3/services/compute => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 17:28:41.342702 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06db4da0-c6cc-4773-9986-adfaba99d9ea None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:41.344902 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1e5dc2-f45b-4ece-994a-e3dab43260ce None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:41.379723 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e56a10-b0de-4964-8d4e-0ee526e4929e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:41.380404 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e56a10-b0de-4964-8d4e-0ee526e4929e None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:41.380723 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e56a10-b0de-4964-8d4e-0ee526e4929e None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:41.380998 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e56a10-b0de-4964-8d4e-0ee526e4929e None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:41.381590 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e56a10-b0de-4964-8d4e-0ee526e4929e None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:41.382967 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e56a10-b0de-4964-8d4e-0ee526e4929e None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:41.392523 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application [None req-b1e56a10-b0de-4964-8d4e-0ee526e4929e None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 17:28:41.392523 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:28:41.392523 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 17:28:41.392523 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 17:28:41.392523 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 17:28:41.392523 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 17:28:41.392523 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 17:28:41.392523 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:28:41.392523 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 17:28:41.392523 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Aug 24 17:28:41.392523 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 17:28:41.392523 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Aug 24 17:28:41.392523 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:28:41.392523 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:28:41.392523 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:28:41.392523 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:28:41.392523 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:28:41.393551 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:28:41.393551 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:28:41.393551 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:28:41.393551 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:28:41.393551 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:28:41.393551 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:28:41.393551 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 17:28:41.393551 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 17:28:41.393551 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 17:28:41.393551 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 17:28:41.393551 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:28:41.393551 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:28:41.393551 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:28:41.393551 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:28:41.393551 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:28:41.393551 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:28:41.394461 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:28:41.394461 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application with Lock( Aug 24 17:28:41.394461 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:28:41.394461 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:28:41.394461 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:28:41.394461 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:28:41.394461 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:28:41.394461 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:28:41.394461 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:28:41.394461 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:28:41.394461 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 17:28:41.394461 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:28:41.394461 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 17:28:41.394461 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Aug 24 17:28:41.394461 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 480/959] 10.222.0.45 () {56 vars in 1126 bytes} [Wed Aug 24 17:28:41 2022] GET /identity/v3/services/object-store => generated 93 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:28:41.395258 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e56a10-b0de-4964-8d4e-0ee526e4929e None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:41.423703 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:41.451396 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06db4da0-c6cc-4773-9986-adfaba99d9ea None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:41.488336 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57d415c-b64f-458e-b05e-69f4dd719edb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:41.488997 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57d415c-b64f-458e-b05e-69f4dd719edb None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:41.489277 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57d415c-b64f-458e-b05e-69f4dd719edb None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:41.489557 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57d415c-b64f-458e-b05e-69f4dd719edb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:41.500524 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57d415c-b64f-458e-b05e-69f4dd719edb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:41.502063 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57d415c-b64f-458e-b05e-69f4dd719edb None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:41.502638 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e56a10-b0de-4964-8d4e-0ee526e4929e None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:41.516833 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application [None req-a57d415c-b64f-458e-b05e-69f4dd719edb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:28:41.516833 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:28:41.516833 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:28:41.516833 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:28:41.516833 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:28:41.516833 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:28:41.516833 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:28:41.516833 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:28:41.516833 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:28:41.516833 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:28:41.516833 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:28:41.516833 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:28:41.516833 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:28:41.516833 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:28:41.516833 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:28:41.516833 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:28:41.517825 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:28:41.517825 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:28:41.517825 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:28:41.517825 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:28:41.517825 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:28:41.517825 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:28:41.517825 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:28:41.517825 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application with Lock( Aug 24 17:28:41.517825 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:28:41.517825 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:28:41.517825 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:28:41.517825 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:28:41.517825 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:28:41.517825 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:28:41.517825 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:28:41.517825 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:28:41.517825 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:28:41.518711 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:28:41.518711 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:28:41.518711 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Aug 24 17:28:41.518711 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 480/960] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 17:28:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:28:41.518711 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57d415c-b64f-458e-b05e-69f4dd719edb None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:41.523193 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:41.544496 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581c50ea-b810-4308-854b-ce938dcac916 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:41.545201 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581c50ea-b810-4308-854b-ce938dcac916 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:41.545476 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581c50ea-b810-4308-854b-ce938dcac916 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:41.545744 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581c50ea-b810-4308-854b-ce938dcac916 None admin] PATH_INFO: `/v3/services` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:41.546369 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581c50ea-b810-4308-854b-ce938dcac916 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:41.546666 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581c50ea-b810-4308-854b-ce938dcac916 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:41.548033 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581c50ea-b810-4308-854b-ce938dcac916 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:41.553361 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 481/961] 10.222.0.45 () {56 vars in 1112 bytes} [Wed Aug 24 17:28:41 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:41.554074 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581c50ea-b810-4308-854b-ce938dcac916 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:41.630549 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57d415c-b64f-458e-b05e-69f4dd719edb None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:41.657576 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:41.669302 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581c50ea-b810-4308-854b-ce938dcac916 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:41.675658 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccf4af6-df2a-48b5-a202-64db4541b3c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:41.676211 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccf4af6-df2a-48b5-a202-64db4541b3c1 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:41.676497 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccf4af6-df2a-48b5-a202-64db4541b3c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:41.677240 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccf4af6-df2a-48b5-a202-64db4541b3c1 None admin] PATH_INFO: `/v3/services` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:41.677886 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccf4af6-df2a-48b5-a202-64db4541b3c1 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:41.678195 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccf4af6-df2a-48b5-a202-64db4541b3c1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:41.679948 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccf4af6-df2a-48b5-a202-64db4541b3c1 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:41.690786 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 481/962] 10.222.0.45 () {56 vars in 1122 bytes} [Wed Aug 24 17:28:41 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:41.690786 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccf4af6-df2a-48b5-a202-64db4541b3c1 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:41.711176 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76ba2c3-7f66-435a-8fed-76af0ffbd04f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:41.711801 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76ba2c3-7f66-435a-8fed-76af0ffbd04f None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:41.712080 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76ba2c3-7f66-435a-8fed-76af0ffbd04f None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:41.712348 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76ba2c3-7f66-435a-8fed-76af0ffbd04f None admin] PATH_INFO: `/v3/domains` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:41.713020 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76ba2c3-7f66-435a-8fed-76af0ffbd04f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:41.713358 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76ba2c3-7f66-435a-8fed-76af0ffbd04f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:41.714765 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76ba2c3-7f66-435a-8fed-76af0ffbd04f None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:41.726690 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 482/963] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 24 17:28:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:41.732484 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76ba2c3-7f66-435a-8fed-76af0ffbd04f None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:41.762908 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:41.805576 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccf4af6-df2a-48b5-a202-64db4541b3c1 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:41.842925 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f2791c-6388-4ef3-b724-3a05264a4cca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:41.843535 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f2791c-6388-4ef3-b724-3a05264a4cca None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:41.843819 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f2791c-6388-4ef3-b724-3a05264a4cca None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:41.844090 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f2791c-6388-4ef3-b724-3a05264a4cca None admin] PATH_INFO: `/v3/services` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:41.844865 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f2791c-6388-4ef3-b724-3a05264a4cca None admin] RBAC: Adding query filter params (type=compute) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:41.845172 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f2791c-6388-4ef3-b724-3a05264a4cca None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:41.846542 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f2791c-6388-4ef3-b724-3a05264a4cca None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:41.847122 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76ba2c3-7f66-435a-8fed-76af0ffbd04f None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:41.851487 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 482/964] 10.222.0.45 () {56 vars in 1112 bytes} [Wed Aug 24 17:28:41 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:41.852109 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f2791c-6388-4ef3-b724-3a05264a4cca None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:41.902991 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470a3e0d-c78a-4b89-b571-5baa53e78aff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:41.904331 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470a3e0d-c78a-4b89-b571-5baa53e78aff None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:41.904897 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470a3e0d-c78a-4b89-b571-5baa53e78aff None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:41.905678 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470a3e0d-c78a-4b89-b571-5baa53e78aff None admin] PATH_INFO: `/v3/services` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:41.907211 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470a3e0d-c78a-4b89-b571-5baa53e78aff None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:41.908083 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470a3e0d-c78a-4b89-b571-5baa53e78aff None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:41.910164 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470a3e0d-c78a-4b89-b571-5baa53e78aff None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:41.919127 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 483/965] 10.222.0.45 () {56 vars in 1122 bytes} [Wed Aug 24 17:28:41 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:41.924787 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470a3e0d-c78a-4b89-b571-5baa53e78aff None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:41.943694 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:41.969899 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f2791c-6388-4ef3-b724-3a05264a4cca None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:42.006578 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:42.014711 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743bc81d-0790-428f-aa12-9148cbedc396 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:42.015429 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743bc81d-0790-428f-aa12-9148cbedc396 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:42.015817 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743bc81d-0790-428f-aa12-9148cbedc396 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:42.016163 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743bc81d-0790-428f-aa12-9148cbedc396 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:42.024893 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743bc81d-0790-428f-aa12-9148cbedc396 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:42.026906 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743bc81d-0790-428f-aa12-9148cbedc396 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:42.039451 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application [None req-743bc81d-0790-428f-aa12-9148cbedc396 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 17:28:42.039451 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:28:42.039451 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:28:42.039451 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:28:42.039451 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:28:42.039451 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:28:42.039451 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:28:42.039451 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:28:42.039451 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:28:42.039451 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:28:42.039451 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:28:42.039451 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:28:42.039451 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 17:28:42.039451 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 17:28:42.039451 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 17:28:42.039451 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 17:28:42.040429 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:28:42.040429 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:28:42.040429 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 17:28:42.040429 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:28:42.040429 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 17:28:42.040429 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:28:42.040429 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:28:42.040429 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:28:42.040429 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:28:42.040429 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:28:42.040429 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:28:42.040429 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application with Lock( Aug 24 17:28:42.040429 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:28:42.040429 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:28:42.040429 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:28:42.040429 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:28:42.040429 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:28:42.040429 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:28:42.041445 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:28:42.041445 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:28:42.041445 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 17:28:42.041445 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 17:28:42.041445 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 17:28:42.041445 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 17:28:42.041445 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 17:28:42.041445 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 17:28:42.041445 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 17:28:42.041445 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Aug 24 17:28:42.041445 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 483/966] 10.222.0.45 () {56 vars in 1117 bytes} [Wed Aug 24 17:28:41 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:28:42.046490 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743bc81d-0790-428f-aa12-9148cbedc396 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:42.059593 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470a3e0d-c78a-4b89-b571-5baa53e78aff None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:42.100496 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5daf6c4-0ecc-4a2e-8656-99d82706f32d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:42.101421 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5daf6c4-0ecc-4a2e-8656-99d82706f32d None admin] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:42.101773 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5daf6c4-0ecc-4a2e-8656-99d82706f32d None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:42.102121 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5daf6c4-0ecc-4a2e-8656-99d82706f32d None admin] PATH_INFO: `/v3/endpoints` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:42.103023 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5daf6c4-0ecc-4a2e-8656-99d82706f32d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:42.104937 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5daf6c4-0ecc-4a2e-8656-99d82706f32d None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:42.140683 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 484/967] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 17:28:41 2022] POST /identity/v3/endpoints => generated 349 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:28:42.172205 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743bc81d-0790-428f-aa12-9148cbedc396 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:42.219855 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57581968-a735-4c9e-80d3-fe3a34fd9532 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:42.220863 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57581968-a735-4c9e-80d3-fe3a34fd9532 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:42.221331 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57581968-a735-4c9e-80d3-fe3a34fd9532 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:42.221843 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57581968-a735-4c9e-80d3-fe3a34fd9532 None admin] PATH_INFO: `/v3/users` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:42.222793 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57581968-a735-4c9e-80d3-fe3a34fd9532 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:42.223295 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57581968-a735-4c9e-80d3-fe3a34fd9532 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:42.224990 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57581968-a735-4c9e-80d3-fe3a34fd9532 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:42.246832 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 484/968] 10.222.0.45 () {56 vars in 1149 bytes} [Wed Aug 24 17:28:42 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:28:42.253728 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5daf6c4-0ecc-4a2e-8656-99d82706f32d None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:42.338121 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:42.374431 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5daf6c4-0ecc-4a2e-8656-99d82706f32d None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:42.413148 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f1ff42-d301-41bd-8fd0-28ad8e725400 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:42.413781 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f1ff42-d301-41bd-8fd0-28ad8e725400 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:42.414064 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f1ff42-d301-41bd-8fd0-28ad8e725400 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:42.414340 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f1ff42-d301-41bd-8fd0-28ad8e725400 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:42.485746 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f1ff42-d301-41bd-8fd0-28ad8e725400 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:42.487126 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f1ff42-d301-41bd-8fd0-28ad8e725400 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:42.562489 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:42.598183 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 485/969] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Aug 24 17:28:42 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 17:28:42.605625 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57581968-a735-4c9e-80d3-fe3a34fd9532 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:42.654833 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:42.710714 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57581968-a735-4c9e-80d3-fe3a34fd9532 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:42.730788 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f7dd5a-12c8-44d6-8e07-511d10c4481c None None] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:42.731115 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f7dd5a-12c8-44d6-8e07-511d10c4481c None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:42.731612 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f7dd5a-12c8-44d6-8e07-511d10c4481c None None] PATH_INFO: `/` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:42.733236 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 486/970] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:28:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 17:28:42.738505 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08253ce9-baa3-4412-8714-29e50f700cd3 None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:42.738736 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08253ce9-baa3-4412-8714-29e50f700cd3 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:42.739004 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08253ce9-baa3-4412-8714-29e50f700cd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:42.747075 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19eb33f2-4099-4f35-bd62-93c8ba431135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:42.747763 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19eb33f2-4099-4f35-bd62-93c8ba431135 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:42.748050 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19eb33f2-4099-4f35-bd62-93c8ba431135 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:42.748320 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19eb33f2-4099-4f35-bd62-93c8ba431135 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:42.758154 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-08253ce9-baa3-4412-8714-29e50f700cd3 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:28:42.759034 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19eb33f2-4099-4f35-bd62-93c8ba431135 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:42.759890 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-08253ce9-baa3-4412-8714-29e50f700cd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:28:42.761157 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19eb33f2-4099-4f35-bd62-93c8ba431135 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:42.780587 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application [None req-19eb33f2-4099-4f35-bd62-93c8ba431135 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:28:42.780587 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:28:42.780587 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:28:42.780587 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:28:42.780587 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:28:42.780587 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:28:42.780587 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:28:42.780587 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:28:42.780587 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:28:42.780587 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:28:42.780587 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:28:42.780587 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:28:42.780587 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:28:42.780587 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:28:42.780587 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:28:42.780587 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:28:42.781575 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:28:42.781575 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:28:42.781575 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:28:42.781575 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:28:42.781575 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:28:42.781575 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:28:42.781575 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:28:42.781575 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application with Lock( Aug 24 17:28:42.781575 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:28:42.781575 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:28:42.781575 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:28:42.781575 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:28:42.781575 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:28:42.781575 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:28:42.781575 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:28:42.781575 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:28:42.781575 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:28:42.782463 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:28:42.782463 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:28:42.782463 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Aug 24 17:28:42.782463 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 485/971] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 17:28:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:28:42.784045 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19eb33f2-4099-4f35-bd62-93c8ba431135 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:42.848824 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 487/972] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:28:42 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 17:28:42.888735 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:42.893165 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19eb33f2-4099-4f35-bd62-93c8ba431135 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:42.928982 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6705f8-9043-4796-92b2-fd9780847d2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:42.929598 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6705f8-9043-4796-92b2-fd9780847d2a None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:42.929876 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6705f8-9043-4796-92b2-fd9780847d2a None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:42.930147 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6705f8-9043-4796-92b2-fd9780847d2a None admin] PATH_INFO: `/v3/domains` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:42.930785 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6705f8-9043-4796-92b2-fd9780847d2a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:42.931088 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6705f8-9043-4796-92b2-fd9780847d2a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:42.932496 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6705f8-9043-4796-92b2-fd9780847d2a None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:42.945074 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 486/973] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 24 17:28:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:42.951298 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08253ce9-baa3-4412-8714-29e50f700cd3 None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:43.056749 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:43.062139 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08253ce9-baa3-4412-8714-29e50f700cd3 None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:43.103470 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9cef53-eb3b-4909-87d6-ccb4bc1cdf72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:43.104915 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9cef53-eb3b-4909-87d6-ccb4bc1cdf72 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:43.105368 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9cef53-eb3b-4909-87d6-ccb4bc1cdf72 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:43.105659 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9cef53-eb3b-4909-87d6-ccb4bc1cdf72 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:43.113863 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9cef53-eb3b-4909-87d6-ccb4bc1cdf72 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:43.115304 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9cef53-eb3b-4909-87d6-ccb4bc1cdf72 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:43.124948 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6705f8-9043-4796-92b2-fd9780847d2a None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:43.130404 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application [None req-af9cef53-eb3b-4909-87d6-ccb4bc1cdf72 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:28:43.130404 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:28:43.130404 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:28:43.130404 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:28:43.130404 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:28:43.130404 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:28:43.130404 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:28:43.130404 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:28:43.130404 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:28:43.130404 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:28:43.130404 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:28:43.130404 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:28:43.130404 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 17:28:43.130404 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 17:28:43.130404 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 17:28:43.131309 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 17:28:43.131309 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:28:43.131309 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:28:43.131309 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:28:43.131309 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:28:43.131309 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:28:43.131309 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:28:43.131309 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:28:43.131309 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application with Lock( Aug 24 17:28:43.131309 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:28:43.131309 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:28:43.131309 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:28:43.131309 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:28:43.131309 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:28:43.131309 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:28:43.131309 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:28:43.131309 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:28:43.132200 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 17:28:43.132200 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 17:28:43.132200 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 17:28:43.132200 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 17:28:43.132200 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 17:28:43.132200 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 17:28:43.132200 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:28:43.132200 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Aug 24 17:28:43.132200 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 488/974] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Aug 24 17:28:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:28:43.139785 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9cef53-eb3b-4909-87d6-ccb4bc1cdf72 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:43.238834 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6705f8-9043-4796-92b2-fd9780847d2a None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:43.259456 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9cef53-eb3b-4909-87d6-ccb4bc1cdf72 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:43.283672 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a58514-ae27-4c2e-a5cf-495efe703eaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:43.284708 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a58514-ae27-4c2e-a5cf-495efe703eaf None admin] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:43.285183 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a58514-ae27-4c2e-a5cf-495efe703eaf None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:43.285638 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a58514-ae27-4c2e-a5cf-495efe703eaf None admin] PATH_INFO: `/v3/users` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:43.287025 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a58514-ae27-4c2e-a5cf-495efe703eaf None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:43.289188 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a58514-ae27-4c2e-a5cf-495efe703eaf None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:43.303781 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949d402a-27c0-45d6-92ba-3a628278a187 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:43.304407 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949d402a-27c0-45d6-92ba-3a628278a187 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:43.304710 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949d402a-27c0-45d6-92ba-3a628278a187 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:43.304956 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949d402a-27c0-45d6-92ba-3a628278a187 None admin] PATH_INFO: `/v3/projects` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:43.305658 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949d402a-27c0-45d6-92ba-3a628278a187 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:43.305996 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949d402a-27c0-45d6-92ba-3a628278a187 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:43.307669 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.sql.core [None req-12a58514-ae27-4c2e-a5cf-495efe703eaf None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 24 17:28:43.307669 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 24 17:28:43.307669 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [parameters: {'user_id': 'd223a4b6710b4afeb365f42e1cb6cc11', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 24 17:28:43.307669 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81758) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 24 17:28:43.307981 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949d402a-27c0-45d6-92ba-3a628278a187 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:43.314837 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application [None req-12a58514-ae27-4c2e-a5cf-495efe703eaf None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 24 17:28:43.314837 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:28:43.314837 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 17:28:43.314837 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 17:28:43.314837 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 17:28:43.314837 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 17:28:43.314837 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 17:28:43.314837 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 17:28:43.314837 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 17:28:43.314837 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application conn.query(q) Aug 24 17:28:43.314837 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 17:28:43.314837 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 17:28:43.314837 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 17:28:43.314837 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application result.read() Aug 24 17:28:43.314837 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 17:28:43.315747 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 17:28:43.315747 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 17:28:43.315747 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 17:28:43.315747 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 17:28:43.315747 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 17:28:43.315747 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 17:28:43.315747 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 17:28:43.315747 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 24 17:28:43.315747 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Aug 24 17:28:43.315747 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 24 17:28:43.315747 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Aug 24 17:28:43.315747 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:28:43.315747 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 24 17:28:43.315747 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 24 17:28:43.315747 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Aug 24 17:28:43.315747 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Aug 24 17:28:43.315747 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 17:28:43.315747 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application next(self.gen) Aug 24 17:28:43.316752 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 24 17:28:43.316752 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application yield resource Aug 24 17:28:43.316752 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 17:28:43.316752 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application next(self.gen) Aug 24 17:28:43.316752 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 24 17:28:43.316752 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application self.session.rollback() Aug 24 17:28:43.316752 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 24 17:28:43.316752 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application self.force_reraise() Aug 24 17:28:43.316752 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 24 17:28:43.316752 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application raise self.value Aug 24 17:28:43.316752 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 24 17:28:43.316752 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 24 17:28:43.316752 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 24 17:28:43.316752 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application session.commit() Aug 24 17:28:43.316752 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 24 17:28:43.316752 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 24 17:28:43.316752 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 24 17:28:43.317695 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application self._prepare_impl() Aug 24 17:28:43.317695 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 24 17:28:43.317695 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application self.session.flush() Aug 24 17:28:43.317695 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 24 17:28:43.317695 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application self._flush(objects) Aug 24 17:28:43.317695 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 24 17:28:43.317695 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 24 17:28:43.317695 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 24 17:28:43.317695 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application compat.raise_( Aug 24 17:28:43.317695 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 17:28:43.317695 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application raise exception Aug 24 17:28:43.317695 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 24 17:28:43.317695 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application flush_context.execute() Aug 24 17:28:43.317695 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 24 17:28:43.317695 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application rec.execute(self) Aug 24 17:28:43.317695 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 24 17:28:43.317695 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 24 17:28:43.318601 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 24 17:28:43.318601 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 24 17:28:43.318601 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Aug 24 17:28:43.318601 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application result = connection._execute_20( Aug 24 17:28:43.318601 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 24 17:28:43.318601 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 24 17:28:43.318601 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 24 17:28:43.318601 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 24 17:28:43.318601 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 24 17:28:43.318601 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 24 17:28:43.318601 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 24 17:28:43.318601 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 24 17:28:43.318601 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 24 17:28:43.318601 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 24 17:28:43.318601 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 17:28:43.318601 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application raise exception Aug 24 17:28:43.319477 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 17:28:43.319477 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 17:28:43.319477 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 17:28:43.319477 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 17:28:43.319477 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 17:28:43.319477 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 17:28:43.319477 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 17:28:43.319477 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application conn.query(q) Aug 24 17:28:43.319477 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 17:28:43.319477 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 17:28:43.319477 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 17:28:43.319477 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application result.read() Aug 24 17:28:43.319477 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 17:28:43.319477 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 17:28:43.319477 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 17:28:43.319477 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 17:28:43.320335 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 17:28:43.320335 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 17:28:43.320335 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 17:28:43.320335 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 17:28:43.320335 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 24 17:28:43.320335 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 24 17:28:43.320335 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application [parameters: {'user_id': 'd223a4b6710b4afeb365f42e1cb6cc11', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 24 17:28:43.320335 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 24 17:28:43.320335 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Aug 24 17:28:43.320335 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 17:28:43.320335 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Aug 24 17:28:43.320335 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:28:43.320335 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:28:43.320335 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:28:43.320335 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:28:43.323896 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:28:43.323896 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:28:43.323896 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:28:43.323896 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:28:43.323896 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:28:43.323896 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:28:43.323896 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:28:43.323896 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Aug 24 17:28:43.323896 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Aug 24 17:28:43.323896 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:28:43.323896 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:28:43.323896 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 17:28:43.323896 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:28:43.323896 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 17:28:43.323896 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:28:43.323896 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Aug 24 17:28:43.323896 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Aug 24 17:28:43.326218 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Aug 24 17:28:43.326218 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Aug 24 17:28:43.326218 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 24 17:28:43.326218 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 24 17:28:43.326218 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 24 17:28:43.326218 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Aug 24 17:28:43.328415 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 489/975] 10.222.0.45 () {56 vars in 1148 bytes} [Wed Aug 24 17:28:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:28:43.328415 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949d402a-27c0-45d6-92ba-3a628278a187 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:43.329040 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 487/976] 10.222.0.45 () {60 vars in 1131 bytes} [Wed Aug 24 17:28:43 2022] POST /identity/v3/users => generated 160 bytes in 204 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:28:43.340625 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a58514-ae27-4c2e-a5cf-495efe703eaf None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:43.351207 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:43.405086 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:43.454718 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949d402a-27c0-45d6-92ba-3a628278a187 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:43.458422 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a58514-ae27-4c2e-a5cf-495efe703eaf None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:43.492531 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726491b0-2b21-4b38-bb2c-ba65b883ff70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:43.493437 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726491b0-2b21-4b38-bb2c-ba65b883ff70 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:43.493864 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726491b0-2b21-4b38-bb2c-ba65b883ff70 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:43.494283 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726491b0-2b21-4b38-bb2c-ba65b883ff70 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:43.495448 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6388f385-3e95-4a77-b1e7-e1be2d66f302 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:43.495979 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726491b0-2b21-4b38-bb2c-ba65b883ff70 None admin] RBAC: Adding query filter params (role.id=d6d7b68e2dee4176bb8712ffbac4808f, scope.project.id=2ad8a32bd9d2434097bd08b111914ca1, user.id=ea143d042ee94ae7a4544d90d18db7db) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:43.496169 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6388f385-3e95-4a77-b1e7-e1be2d66f302 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:43.496446 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6388f385-3e95-4a77-b1e7-e1be2d66f302 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:43.496760 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6388f385-3e95-4a77-b1e7-e1be2d66f302 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:43.496945 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726491b0-2b21-4b38-bb2c-ba65b883ff70 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:43.498550 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726491b0-2b21-4b38-bb2c-ba65b883ff70 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:43.505353 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6388f385-3e95-4a77-b1e7-e1be2d66f302 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:43.505781 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 490/977] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Aug 24 17:28:43 2022] GET /identity/v3/role_assignments?user.id=ea143d042ee94ae7a4544d90d18db7db&scope.project.id=2ad8a32bd9d2434097bd08b111914ca1&role.id=d6d7b68e2dee4176bb8712ffbac4808f => generated 608 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:28:43.508093 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6388f385-3e95-4a77-b1e7-e1be2d66f302 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:43.533744 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application [None req-6388f385-3e95-4a77-b1e7-e1be2d66f302 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 17:28:43.533744 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:28:43.533744 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:28:43.533744 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:28:43.533744 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:28:43.533744 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:28:43.533744 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:28:43.533744 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:28:43.533744 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:28:43.533744 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:28:43.533744 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:28:43.533744 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:28:43.533744 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 17:28:43.533744 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 17:28:43.533744 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 17:28:43.533744 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 17:28:43.534955 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:28:43.534955 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:28:43.534955 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 17:28:43.534955 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:28:43.534955 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 17:28:43.534955 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:28:43.534955 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:28:43.534955 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:28:43.534955 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:28:43.534955 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:28:43.534955 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:28:43.534955 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application with Lock( Aug 24 17:28:43.534955 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:28:43.534955 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:28:43.534955 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:28:43.534955 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:28:43.534955 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:28:43.534955 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:28:43.536122 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:28:43.536122 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:28:43.536122 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 17:28:43.536122 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 17:28:43.536122 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 17:28:43.536122 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 17:28:43.536122 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 17:28:43.536122 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 17:28:43.536122 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 17:28:43.536122 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Aug 24 17:28:43.536122 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 488/978] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Aug 24 17:28:43 2022] GET /identity/v3/users/cinder => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 17:28:43.542023 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726491b0-2b21-4b38-bb2c-ba65b883ff70 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:43.636602 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:43.655560 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726491b0-2b21-4b38-bb2c-ba65b883ff70 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:43.696063 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136391e5-fa9f-49f5-ad85-bdf2fcd22ea7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:43.696719 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136391e5-fa9f-49f5-ad85-bdf2fcd22ea7 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:43.697000 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136391e5-fa9f-49f5-ad85-bdf2fcd22ea7 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:43.697277 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136391e5-fa9f-49f5-ad85-bdf2fcd22ea7 None admin] PATH_INFO: `/v3/users` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:43.698521 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136391e5-fa9f-49f5-ad85-bdf2fcd22ea7 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:43.698825 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136391e5-fa9f-49f5-ad85-bdf2fcd22ea7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:43.700274 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136391e5-fa9f-49f5-ad85-bdf2fcd22ea7 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:43.737526 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 491/979] 10.222.0.45 () {56 vars in 1101 bytes} [Wed Aug 24 17:28:43 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:44.023994 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c5cf4d-9a66-42f5-9731-0b4327c3ae6d None None] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:44.024515 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c5cf4d-9a66-42f5-9731-0b4327c3ae6d None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:44.024618 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c5cf4d-9a66-42f5-9731-0b4327c3ae6d None None] PATH_INFO: `/` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:44.029326 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 489/980] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:28:44 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:28:44.034625 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758ec2e2-8613-4d9c-aabf-9101a4dcc484 None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:44.035029 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758ec2e2-8613-4d9c-aabf-9101a4dcc484 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:44.035216 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758ec2e2-8613-4d9c-aabf-9101a4dcc484 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:44.053923 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-758ec2e2-8613-4d9c-aabf-9101a4dcc484 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:28:44.055117 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-758ec2e2-8613-4d9c-aabf-9101a4dcc484 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:28:44.125555 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 492/981] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:28:44 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:28:44.292136 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba01fa0c-c549-47c5-ad32-7ebd784479b8 None None] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:44.292430 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba01fa0c-c549-47c5-ad32-7ebd784479b8 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:44.292644 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba01fa0c-c549-47c5-ad32-7ebd784479b8 None None] PATH_INFO: `/` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:44.294129 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 490/982] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:28:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:28:44.299238 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516a554b-a231-4d23-b52a-d3b0ebecd2a2 None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:44.299537 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516a554b-a231-4d23-b52a-d3b0ebecd2a2 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:44.299818 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516a554b-a231-4d23-b52a-d3b0ebecd2a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:44.317750 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-516a554b-a231-4d23-b52a-d3b0ebecd2a2 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:28:44.318950 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-516a554b-a231-4d23-b52a-d3b0ebecd2a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:28:44.386555 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 493/983] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:28:44 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:28:44.391611 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba01fa0c-c549-47c5-ad32-7ebd784479b8 None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:44.476298 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:44.497604 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba01fa0c-c549-47c5-ad32-7ebd784479b8 None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:44.531114 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1101399-8e63-40f1-8a77-555a3513d6c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:44.531721 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1101399-8e63-40f1-8a77-555a3513d6c3 None admin] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:44.532003 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1101399-8e63-40f1-8a77-555a3513d6c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:44.532281 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1101399-8e63-40f1-8a77-555a3513d6c3 None admin] PATH_INFO: `/v3/services` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:44.532969 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1101399-8e63-40f1-8a77-555a3513d6c3 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:44.534345 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1101399-8e63-40f1-8a77-555a3513d6c3 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:44.552675 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 491/984] 10.222.0.45 () {60 vars in 1141 bytes} [Wed Aug 24 17:28:44 2022] POST /identity/v3/services => generated 242 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:28:44.658071 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516a554b-a231-4d23-b52a-d3b0ebecd2a2 None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:44.660391 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:44.765502 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516a554b-a231-4d23-b52a-d3b0ebecd2a2 None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:44.813154 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb67f9fb-739a-41f6-a8ce-d59eb7a62084 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:44.813720 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb67f9fb-739a-41f6-a8ce-d59eb7a62084 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:44.814002 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb67f9fb-739a-41f6-a8ce-d59eb7a62084 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:44.814274 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb67f9fb-739a-41f6-a8ce-d59eb7a62084 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:44.814855 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb67f9fb-739a-41f6-a8ce-d59eb7a62084 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:44.816231 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb67f9fb-739a-41f6-a8ce-d59eb7a62084 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:44.825600 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application [None req-fb67f9fb-739a-41f6-a8ce-d59eb7a62084 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 17:28:44.825600 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:28:44.825600 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 17:28:44.825600 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 17:28:44.825600 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 17:28:44.825600 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 17:28:44.825600 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 17:28:44.825600 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:28:44.825600 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 17:28:44.825600 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Aug 24 17:28:44.825600 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 17:28:44.825600 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Aug 24 17:28:44.825600 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:28:44.825600 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:28:44.825600 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:28:44.825600 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:28:44.825600 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:28:44.826637 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:28:44.826637 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:28:44.826637 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:28:44.826637 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:28:44.826637 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:28:44.826637 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:28:44.826637 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 17:28:44.826637 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 17:28:44.826637 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 17:28:44.826637 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 17:28:44.826637 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:28:44.826637 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:28:44.826637 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:28:44.826637 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:28:44.826637 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:28:44.826637 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:28:44.828135 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:28:44.828135 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application with Lock( Aug 24 17:28:44.828135 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:28:44.828135 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:28:44.828135 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:28:44.828135 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:28:44.828135 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:28:44.828135 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:28:44.828135 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:28:44.828135 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:28:44.828135 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 17:28:44.828135 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:28:44.828135 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 17:28:44.828135 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Aug 24 17:28:44.828135 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 494/985] 10.222.0.45 () {56 vars in 1126 bytes} [Wed Aug 24 17:28:44 2022] GET /identity/v3/services/object-store => generated 93 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:28:44.833640 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1101399-8e63-40f1-8a77-555a3513d6c3 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:44.936822 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1101399-8e63-40f1-8a77-555a3513d6c3 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:44.970355 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ced88b-36a8-4029-bdd4-e658e1ccc6b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:44.970968 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ced88b-36a8-4029-bdd4-e658e1ccc6b2 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:44.971243 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ced88b-36a8-4029-bdd4-e658e1ccc6b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:44.971513 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ced88b-36a8-4029-bdd4-e658e1ccc6b2 None admin] PATH_INFO: `/v3/services` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:44.972143 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ced88b-36a8-4029-bdd4-e658e1ccc6b2 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:44.972444 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ced88b-36a8-4029-bdd4-e658e1ccc6b2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:44.973842 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ced88b-36a8-4029-bdd4-e658e1ccc6b2 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:44.979683 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 492/986] 10.222.0.45 () {56 vars in 1122 bytes} [Wed Aug 24 17:28:44 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:44.985765 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb67f9fb-739a-41f6-a8ce-d59eb7a62084 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:45.048430 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:45.089688 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb67f9fb-739a-41f6-a8ce-d59eb7a62084 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:45.122990 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244b27a9-9274-437b-8e45-f8cf2640133b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:45.123593 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244b27a9-9274-437b-8e45-f8cf2640133b None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:45.123869 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244b27a9-9274-437b-8e45-f8cf2640133b None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:45.124141 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244b27a9-9274-437b-8e45-f8cf2640133b None admin] PATH_INFO: `/v3/services` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:45.124785 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244b27a9-9274-437b-8e45-f8cf2640133b None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:45.125087 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244b27a9-9274-437b-8e45-f8cf2640133b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:45.126449 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244b27a9-9274-437b-8e45-f8cf2640133b None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:45.136779 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 495/987] 10.222.0.45 () {56 vars in 1122 bytes} [Wed Aug 24 17:28:44 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:45.138334 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ced88b-36a8-4029-bdd4-e658e1ccc6b2 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:45.142246 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:45.242216 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ced88b-36a8-4029-bdd4-e658e1ccc6b2 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:45.274862 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5818a932-9418-4a16-ae16-0f43e116c40d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:45.275612 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5818a932-9418-4a16-ae16-0f43e116c40d None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:45.276451 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5818a932-9418-4a16-ae16-0f43e116c40d None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:45.276764 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5818a932-9418-4a16-ae16-0f43e116c40d None admin] PATH_INFO: `/v3/endpoints` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:45.277476 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5818a932-9418-4a16-ae16-0f43e116c40d None admin] RBAC: Adding query filter params (interface=admin, service_id=9d6a35e85e7d471d84818bf5f0795424, region_id=RegionOne) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:45.277786 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5818a932-9418-4a16-ae16-0f43e116c40d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:45.279151 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5818a932-9418-4a16-ae16-0f43e116c40d None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:45.284330 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 493/988] 10.222.0.45 () {56 vars in 1249 bytes} [Wed Aug 24 17:28:45 2022] GET /identity/v3/endpoints?service_id=9d6a35e85e7d471d84818bf5f0795424&interface=admin®ion_id=RegionOne => generated 193 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:45.289705 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244b27a9-9274-437b-8e45-f8cf2640133b None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:45.390885 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244b27a9-9274-437b-8e45-f8cf2640133b None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:45.416919 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:45.427106 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e637bf1b-699d-44d3-a4db-af61e8feb7dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:45.427726 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e637bf1b-699d-44d3-a4db-af61e8feb7dd None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:45.428007 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e637bf1b-699d-44d3-a4db-af61e8feb7dd None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:45.428304 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e637bf1b-699d-44d3-a4db-af61e8feb7dd None admin] PATH_INFO: `/v3/services` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:45.428945 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e637bf1b-699d-44d3-a4db-af61e8feb7dd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:45.430352 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e637bf1b-699d-44d3-a4db-af61e8feb7dd None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:45.444592 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 496/989] 10.222.0.45 () {56 vars in 1087 bytes} [Wed Aug 24 17:28:45 2022] GET /identity/v3/services => generated 1727 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:28:45.519587 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96ddcda-de05-4703-a490-f16d7c120264 None None] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:45.519881 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96ddcda-de05-4703-a490-f16d7c120264 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:45.520159 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96ddcda-de05-4703-a490-f16d7c120264 None None] PATH_INFO: `/` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:45.521590 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 494/990] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:28:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:28:45.527436 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd932911-81b0-4a57-9b4d-57127573b203 None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:45.527734 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd932911-81b0-4a57-9b4d-57127573b203 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:45.528003 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd932911-81b0-4a57-9b4d-57127573b203 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:45.546889 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-bd932911-81b0-4a57-9b4d-57127573b203 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:28:45.548138 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-bd932911-81b0-4a57-9b4d-57127573b203 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:28:45.622872 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 497/991] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:28:45 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:28:45.890464 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96ddcda-de05-4703-a490-f16d7c120264 None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:45.959340 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:45.995507 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96ddcda-de05-4703-a490-f16d7c120264 None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:46.028163 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0219bb0-1e3a-4aaa-9bd0-b1eeb123071b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:46.028817 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0219bb0-1e3a-4aaa-9bd0-b1eeb123071b None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:46.029093 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0219bb0-1e3a-4aaa-9bd0-b1eeb123071b None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:46.029359 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0219bb0-1e3a-4aaa-9bd0-b1eeb123071b None admin] PATH_INFO: `/v3/services/image` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:46.029935 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0219bb0-1e3a-4aaa-9bd0-b1eeb123071b None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:46.031363 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0219bb0-1e3a-4aaa-9bd0-b1eeb123071b None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:46.040150 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application [None req-d0219bb0-1e3a-4aaa-9bd0-b1eeb123071b None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 17:28:46.040150 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:28:46.040150 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 17:28:46.040150 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 17:28:46.040150 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 17:28:46.040150 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 17:28:46.040150 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 17:28:46.040150 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:28:46.040150 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 17:28:46.040150 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Aug 24 17:28:46.040150 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 17:28:46.040150 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Aug 24 17:28:46.040150 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:28:46.040150 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:28:46.040150 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:28:46.040150 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:28:46.040150 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:28:46.041193 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:28:46.041193 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:28:46.041193 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:28:46.041193 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:28:46.041193 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:28:46.041193 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:28:46.041193 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 17:28:46.041193 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 17:28:46.041193 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 17:28:46.041193 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 17:28:46.041193 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:28:46.041193 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:28:46.041193 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:28:46.041193 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:28:46.041193 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:28:46.041193 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:28:46.045412 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:28:46.045412 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application with Lock( Aug 24 17:28:46.045412 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:28:46.045412 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:28:46.045412 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:28:46.045412 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:28:46.045412 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:28:46.045412 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:28:46.045412 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:28:46.045412 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:28:46.045412 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 17:28:46.045412 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:28:46.045412 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 17:28:46.045412 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Aug 24 17:28:46.045412 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 495/992] 10.222.0.45 () {56 vars in 1105 bytes} [Wed Aug 24 17:28:45 2022] GET /identity/v3/services/image => generated 86 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:28:46.048208 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd932911-81b0-4a57-9b4d-57127573b203 None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:46.146745 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd932911-81b0-4a57-9b4d-57127573b203 None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:46.168028 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:46.183163 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5519bf-3f63-415a-b5c7-8d773f00684c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:46.183771 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5519bf-3f63-415a-b5c7-8d773f00684c None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:46.184044 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5519bf-3f63-415a-b5c7-8d773f00684c None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:46.184312 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5519bf-3f63-415a-b5c7-8d773f00684c None admin] PATH_INFO: `/v3/services` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:46.184962 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5519bf-3f63-415a-b5c7-8d773f00684c None admin] RBAC: Adding query filter params (name=image) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:46.185263 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5519bf-3f63-415a-b5c7-8d773f00684c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:46.186619 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5519bf-3f63-415a-b5c7-8d773f00684c None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:46.191535 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 498/993] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 17:28:46 2022] GET /identity/v3/services?name=image => generated 122 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:28:46.196957 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0219bb0-1e3a-4aaa-9bd0-b1eeb123071b None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:46.298414 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0219bb0-1e3a-4aaa-9bd0-b1eeb123071b None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:46.326972 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:46.332773 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94a465f-161b-4d2a-8a32-444e62f6b6cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:46.333382 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94a465f-161b-4d2a-8a32-444e62f6b6cc None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:46.333663 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94a465f-161b-4d2a-8a32-444e62f6b6cc None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:46.333936 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94a465f-161b-4d2a-8a32-444e62f6b6cc None admin] PATH_INFO: `/v3/services` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:46.334569 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94a465f-161b-4d2a-8a32-444e62f6b6cc None admin] RBAC: Adding query filter params (type=image) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:46.334875 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94a465f-161b-4d2a-8a32-444e62f6b6cc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:46.336242 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94a465f-161b-4d2a-8a32-444e62f6b6cc None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:46.348602 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 496/994] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 17:28:46 2022] GET /identity/v3/services?type=image => generated 122 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:28:46.491432 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1526431-1af2-4ddf-b182-d40cdf2d6a82 None None] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:46.491683 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1526431-1af2-4ddf-b182-d40cdf2d6a82 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:46.491920 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1526431-1af2-4ddf-b182-d40cdf2d6a82 None None] PATH_INFO: `/` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:46.493299 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 499/995] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:28:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:28:46.499260 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb175bb6-3da0-4cf2-b8a3-6b7da72cf103 None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:46.499540 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb175bb6-3da0-4cf2-b8a3-6b7da72cf103 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:46.499831 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb175bb6-3da0-4cf2-b8a3-6b7da72cf103 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:46.519433 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-eb175bb6-3da0-4cf2-b8a3-6b7da72cf103 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:28:46.521052 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-eb175bb6-3da0-4cf2-b8a3-6b7da72cf103 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:28:46.588605 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 497/996] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:28:46 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:28:46.851378 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1526431-1af2-4ddf-b182-d40cdf2d6a82 None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:46.951582 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1526431-1af2-4ddf-b182-d40cdf2d6a82 None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:46.985951 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d045425f-b42b-45b1-8ef6-add1f9ba13ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:46.986580 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d045425f-b42b-45b1-8ef6-add1f9ba13ae None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:46.986854 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d045425f-b42b-45b1-8ef6-add1f9ba13ae None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:46.987125 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d045425f-b42b-45b1-8ef6-add1f9ba13ae None admin] PATH_INFO: `/v3/services/compute` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:46.987701 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d045425f-b42b-45b1-8ef6-add1f9ba13ae None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:46.989092 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d045425f-b42b-45b1-8ef6-add1f9ba13ae None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:46.997586 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application [None req-d045425f-b42b-45b1-8ef6-add1f9ba13ae None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 17:28:46.997586 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:28:46.997586 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 17:28:46.997586 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 17:28:46.997586 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 17:28:46.997586 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 17:28:46.997586 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 17:28:46.997586 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:28:46.997586 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 17:28:46.997586 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Aug 24 17:28:46.997586 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 17:28:46.997586 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Aug 24 17:28:46.997586 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:28:46.997586 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:28:46.997586 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:28:46.997586 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:28:46.997586 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:28:46.998550 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:28:46.998550 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:28:46.998550 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:28:46.998550 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:28:46.998550 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:28:46.998550 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:28:46.998550 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 17:28:46.998550 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 17:28:46.998550 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 17:28:46.998550 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 17:28:46.998550 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:28:46.998550 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:28:46.998550 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:28:46.998550 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:28:46.998550 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:28:46.998550 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:28:46.999370 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:28:46.999370 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application with Lock( Aug 24 17:28:46.999370 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:28:46.999370 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:28:46.999370 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:28:46.999370 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:28:46.999370 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:28:46.999370 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:28:46.999370 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:28:46.999370 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:28:46.999370 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 17:28:46.999370 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:28:46.999370 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 17:28:46.999370 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Aug 24 17:28:47.003458 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 500/997] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Aug 24 17:28:46 2022] GET /identity/v3/services/compute => generated 88 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:28:47.004722 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb175bb6-3da0-4cf2-b8a3-6b7da72cf103 None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:47.100400 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb175bb6-3da0-4cf2-b8a3-6b7da72cf103 None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:47.125664 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:47.134789 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d876f622-af22-4ab4-b09c-646aafe9c154 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:47.135553 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d876f622-af22-4ab4-b09c-646aafe9c154 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:47.136007 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d876f622-af22-4ab4-b09c-646aafe9c154 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:47.136456 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d876f622-af22-4ab4-b09c-646aafe9c154 None admin] PATH_INFO: `/v3/services` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:47.137639 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d876f622-af22-4ab4-b09c-646aafe9c154 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:47.138187 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d876f622-af22-4ab4-b09c-646aafe9c154 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:47.140659 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d876f622-af22-4ab4-b09c-646aafe9c154 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:47.150489 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 498/998] 10.222.0.45 () {56 vars in 1112 bytes} [Wed Aug 24 17:28:47 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:47.151966 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d045425f-b42b-45b1-8ef6-add1f9ba13ae None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:47.201363 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:47.252665 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d045425f-b42b-45b1-8ef6-add1f9ba13ae None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:47.285875 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6781f79a-a4ef-41ae-8803-0a692c798a1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:47.286480 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6781f79a-a4ef-41ae-8803-0a692c798a1a None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:47.286755 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6781f79a-a4ef-41ae-8803-0a692c798a1a None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:47.287025 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6781f79a-a4ef-41ae-8803-0a692c798a1a None admin] PATH_INFO: `/v3/services` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:47.287650 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6781f79a-a4ef-41ae-8803-0a692c798a1a None admin] RBAC: Adding query filter params (type=compute) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:47.287950 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6781f79a-a4ef-41ae-8803-0a692c798a1a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:47.289335 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6781f79a-a4ef-41ae-8803-0a692c798a1a None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:47.298940 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 501/999] 10.222.0.45 () {56 vars in 1112 bytes} [Wed Aug 24 17:28:47 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:28:47.299988 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d876f622-af22-4ab4-b09c-646aafe9c154 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:47.347704 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaba8e0c-d04f-489f-ad84-0dcb1c0dc661 None None] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:47.348012 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaba8e0c-d04f-489f-ad84-0dcb1c0dc661 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:47.348282 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaba8e0c-d04f-489f-ad84-0dcb1c0dc661 None None] PATH_INFO: `/` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:47.349802 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 502/1000] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:28:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 17:28:47.354576 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7851ed79-ad09-4caa-8186-945c6696ac30 None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:47.354795 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7851ed79-ad09-4caa-8186-945c6696ac30 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:47.355103 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7851ed79-ad09-4caa-8186-945c6696ac30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:47.372985 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-7851ed79-ad09-4caa-8186-945c6696ac30 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:28:47.374204 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-7851ed79-ad09-4caa-8186-945c6696ac30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:28:47.406241 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d876f622-af22-4ab4-b09c-646aafe9c154 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:47.442189 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4057f8-2bec-4a3e-8494-17a1692edb30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:47.442791 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4057f8-2bec-4a3e-8494-17a1692edb30 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:47.443068 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4057f8-2bec-4a3e-8494-17a1692edb30 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:47.443333 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4057f8-2bec-4a3e-8494-17a1692edb30 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:47.444027 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4057f8-2bec-4a3e-8494-17a1692edb30 None admin] RBAC: Adding query filter params (interface=public, service_id=fe8e48a864574d3cb931cd9489264ee7, region_id=RegionOne) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:47.444333 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4057f8-2bec-4a3e-8494-17a1692edb30 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:47.445714 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4057f8-2bec-4a3e-8494-17a1692edb30 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:47.451084 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 499/1001] 10.222.0.45 () {56 vars in 1251 bytes} [Wed Aug 24 17:28:47 2022] GET /identity/v3/endpoints?service_id=fe8e48a864574d3cb931cd9489264ee7&interface=public®ion_id=RegionOne => generated 194 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:28:47.452855 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 503/1002] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:28:47 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:28:47.456680 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4057f8-2bec-4a3e-8494-17a1692edb30 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:47.516017 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:47.561302 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4057f8-2bec-4a3e-8494-17a1692edb30 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:47.594901 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67987aa7-7a01-4433-89eb-4e3a667a0edd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:47.595501 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67987aa7-7a01-4433-89eb-4e3a667a0edd None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:47.595781 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67987aa7-7a01-4433-89eb-4e3a667a0edd None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:47.596050 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67987aa7-7a01-4433-89eb-4e3a667a0edd None admin] PATH_INFO: `/v3/services` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:47.596678 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67987aa7-7a01-4433-89eb-4e3a667a0edd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:47.598032 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67987aa7-7a01-4433-89eb-4e3a667a0edd None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:47.604295 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 500/1003] 10.222.0.45 () {56 vars in 1087 bytes} [Wed Aug 24 17:28:47 2022] GET /identity/v3/services => generated 1727 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 17:28:47.720359 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7851ed79-ad09-4caa-8186-945c6696ac30 None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:47.767182 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:47.833178 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7851ed79-ad09-4caa-8186-945c6696ac30 None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:47.868341 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b65f54a-1888-4f94-9b56-18c4345b02f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:47.869153 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b65f54a-1888-4f94-9b56-18c4345b02f0 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:47.869428 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b65f54a-1888-4f94-9b56-18c4345b02f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:47.870294 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b65f54a-1888-4f94-9b56-18c4345b02f0 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:47.870879 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b65f54a-1888-4f94-9b56-18c4345b02f0 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:47.872239 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b65f54a-1888-4f94-9b56-18c4345b02f0 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:47.881324 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application [None req-7b65f54a-1888-4f94-9b56-18c4345b02f0 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 17:28:47.881324 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:28:47.881324 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 17:28:47.881324 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 17:28:47.881324 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 17:28:47.881324 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 17:28:47.881324 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 17:28:47.881324 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:28:47.881324 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 17:28:47.881324 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Aug 24 17:28:47.881324 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 17:28:47.881324 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Aug 24 17:28:47.881324 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:28:47.881324 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:28:47.881324 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:28:47.881324 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:28:47.881324 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:28:47.882311 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:28:47.882311 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:28:47.882311 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:28:47.882311 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:28:47.882311 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:28:47.882311 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:28:47.882311 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 17:28:47.882311 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 17:28:47.882311 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 17:28:47.882311 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 17:28:47.882311 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:28:47.882311 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:28:47.882311 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:28:47.882311 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:28:47.882311 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:28:47.882311 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:28:47.883186 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:28:47.883186 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application with Lock( Aug 24 17:28:47.883186 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:28:47.883186 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:28:47.883186 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:28:47.883186 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:28:47.883186 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:28:47.883186 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:28:47.883186 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:28:47.883186 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:28:47.883186 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 17:28:47.883186 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:28:47.883186 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 17:28:47.883186 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Aug 24 17:28:47.888185 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 504/1004] 10.222.0.45 () {56 vars in 1126 bytes} [Wed Aug 24 17:28:47 2022] GET /identity/v3/services/object-store => generated 93 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:28:47.889541 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67987aa7-7a01-4433-89eb-4e3a667a0edd None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:47.990013 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67987aa7-7a01-4433-89eb-4e3a667a0edd None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:48.007360 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:48.025563 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c272d35-56e6-43d2-a026-66405eed796b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:48.026200 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c272d35-56e6-43d2-a026-66405eed796b None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:48.026477 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c272d35-56e6-43d2-a026-66405eed796b None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:48.026750 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c272d35-56e6-43d2-a026-66405eed796b None admin] PATH_INFO: `/v3/services` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:48.027381 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c272d35-56e6-43d2-a026-66405eed796b None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:48.027682 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c272d35-56e6-43d2-a026-66405eed796b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:48.029074 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c272d35-56e6-43d2-a026-66405eed796b None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:48.038519 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 501/1005] 10.222.0.45 () {56 vars in 1122 bytes} [Wed Aug 24 17:28:47 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:48.040065 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b65f54a-1888-4f94-9b56-18c4345b02f0 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:48.140349 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b65f54a-1888-4f94-9b56-18c4345b02f0 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:48.147237 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:48.175539 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296deba3-a943-45e0-aeb8-f39e3a8777d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:48.176473 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296deba3-a943-45e0-aeb8-f39e3a8777d0 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:48.176473 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296deba3-a943-45e0-aeb8-f39e3a8777d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:48.176808 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296deba3-a943-45e0-aeb8-f39e3a8777d0 None admin] PATH_INFO: `/v3/services` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:48.177351 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296deba3-a943-45e0-aeb8-f39e3a8777d0 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:48.177651 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296deba3-a943-45e0-aeb8-f39e3a8777d0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:48.179007 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296deba3-a943-45e0-aeb8-f39e3a8777d0 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:48.184673 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 505/1006] 10.222.0.45 () {56 vars in 1122 bytes} [Wed Aug 24 17:28:48 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:28:48.190048 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c272d35-56e6-43d2-a026-66405eed796b None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:48.256316 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b31a4b-c0bb-4df9-9fe5-fe7b27a1e39d None None] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:48.256508 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b31a4b-c0bb-4df9-9fe5-fe7b27a1e39d None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:48.256818 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b31a4b-c0bb-4df9-9fe5-fe7b27a1e39d None None] PATH_INFO: `/` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:48.258286 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 506/1007] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:28:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:28:48.262999 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9a0901-bbd1-4445-83ec-57ea85647e90 None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:48.263286 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9a0901-bbd1-4445-83ec-57ea85647e90 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:48.263568 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9a0901-bbd1-4445-83ec-57ea85647e90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:48.282183 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-1c9a0901-bbd1-4445-83ec-57ea85647e90 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:28:48.283379 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-1c9a0901-bbd1-4445-83ec-57ea85647e90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:28:48.296007 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c272d35-56e6-43d2-a026-66405eed796b None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:48.330991 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73eb87f9-7c5d-4b91-9bf3-9f9a2583328e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:48.331606 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73eb87f9-7c5d-4b91-9bf3-9f9a2583328e None admin] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:48.331811 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73eb87f9-7c5d-4b91-9bf3-9f9a2583328e None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:48.332047 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73eb87f9-7c5d-4b91-9bf3-9f9a2583328e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:48.332726 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73eb87f9-7c5d-4b91-9bf3-9f9a2583328e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:48.334180 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73eb87f9-7c5d-4b91-9bf3-9f9a2583328e None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:48.352158 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 502/1008] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 17:28:48 2022] POST /identity/v3/endpoints => generated 325 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:28:48.367331 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 507/1009] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:28:48 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 17:28:48.640558 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73eb87f9-7c5d-4b91-9bf3-9f9a2583328e None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:48.688704 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:48.749247 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73eb87f9-7c5d-4b91-9bf3-9f9a2583328e None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:48.788295 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fcbecd-d612-4bb9-8b8a-a383be5999af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:48.788934 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fcbecd-d612-4bb9-8b8a-a383be5999af None admin] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:48.789231 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fcbecd-d612-4bb9-8b8a-a383be5999af None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:48.789492 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fcbecd-d612-4bb9-8b8a-a383be5999af None admin] PATH_INFO: `/v3/services` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:48.790638 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fcbecd-d612-4bb9-8b8a-a383be5999af None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:48.792047 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fcbecd-d612-4bb9-8b8a-a383be5999af None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:48.812508 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 503/1010] 10.222.0.45 () {60 vars in 1141 bytes} [Wed Aug 24 17:28:48 2022] POST /identity/v3/services => generated 242 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:28:49.515395 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d7c5d2-21a7-4d32-8e75-0b1021356b63 None None] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:49.515918 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d7c5d2-21a7-4d32-8e75-0b1021356b63 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:49.515918 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d7c5d2-21a7-4d32-8e75-0b1021356b63 None None] PATH_INFO: `/` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:49.517345 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 508/1011] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:28:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:28:49.522464 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8253da7-2a80-4e25-aed0-b6a1f957c53a None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:49.522851 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8253da7-2a80-4e25-aed0-b6a1f957c53a None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:49.523018 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8253da7-2a80-4e25-aed0-b6a1f957c53a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:49.540668 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-e8253da7-2a80-4e25-aed0-b6a1f957c53a None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:28:49.542361 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-e8253da7-2a80-4e25-aed0-b6a1f957c53a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:28:49.613326 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 504/1012] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:28:49 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:28:49.881692 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d7c5d2-21a7-4d32-8e75-0b1021356b63 None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:49.978840 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:49.987606 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d7c5d2-21a7-4d32-8e75-0b1021356b63 None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:50.020849 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789d369f-41a2-45f2-8cc3-2af5f7677a00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:50.021487 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789d369f-41a2-45f2-8cc3-2af5f7677a00 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:50.021763 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789d369f-41a2-45f2-8cc3-2af5f7677a00 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:50.022035 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789d369f-41a2-45f2-8cc3-2af5f7677a00 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:50.022907 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789d369f-41a2-45f2-8cc3-2af5f7677a00 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:50.024277 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789d369f-41a2-45f2-8cc3-2af5f7677a00 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:50.033423 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application [None req-789d369f-41a2-45f2-8cc3-2af5f7677a00 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 17:28:50.033423 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:28:50.033423 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 17:28:50.033423 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 17:28:50.033423 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 17:28:50.033423 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 17:28:50.033423 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 17:28:50.033423 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:28:50.033423 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 17:28:50.033423 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Aug 24 17:28:50.033423 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 17:28:50.033423 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Aug 24 17:28:50.033423 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:28:50.033423 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:28:50.033423 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:28:50.033423 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:28:50.033423 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:28:50.034391 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:28:50.034391 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:28:50.034391 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:28:50.034391 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:28:50.034391 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:28:50.034391 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:28:50.034391 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 17:28:50.034391 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 17:28:50.034391 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 17:28:50.034391 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 17:28:50.034391 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:28:50.034391 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:28:50.034391 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:28:50.034391 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:28:50.034391 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:28:50.034391 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:28:50.035355 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:28:50.035355 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application with Lock( Aug 24 17:28:50.035355 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:28:50.035355 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:28:50.035355 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:28:50.035355 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:28:50.035355 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:28:50.035355 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:28:50.035355 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:28:50.035355 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:28:50.035355 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 17:28:50.035355 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:28:50.035355 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 17:28:50.035355 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Aug 24 17:28:50.035355 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 509/1013] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Aug 24 17:28:49 2022] GET /identity/v3/services/compute => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 17:28:50.040354 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8253da7-2a80-4e25-aed0-b6a1f957c53a None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:50.068767 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:50.144484 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8253da7-2a80-4e25-aed0-b6a1f957c53a None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:50.179875 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67096907-9780-4eaa-a7ff-8da64827be27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:50.180483 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67096907-9780-4eaa-a7ff-8da64827be27 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:50.180791 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67096907-9780-4eaa-a7ff-8da64827be27 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:50.181070 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67096907-9780-4eaa-a7ff-8da64827be27 None admin] PATH_INFO: `/v3/services` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:50.181690 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67096907-9780-4eaa-a7ff-8da64827be27 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:50.182019 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67096907-9780-4eaa-a7ff-8da64827be27 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:50.183382 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67096907-9780-4eaa-a7ff-8da64827be27 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:50.188357 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 505/1014] 10.222.0.45 () {56 vars in 1112 bytes} [Wed Aug 24 17:28:50 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:50.193885 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789d369f-41a2-45f2-8cc3-2af5f7677a00 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:50.248722 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603ebe5d-538d-4ac1-a217-25f601e0954f None None] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:50.249051 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603ebe5d-538d-4ac1-a217-25f601e0954f None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:50.249285 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603ebe5d-538d-4ac1-a217-25f601e0954f None None] PATH_INFO: `/` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:50.250741 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 506/1015] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:28:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 17:28:50.255461 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d037840a-bcba-43b0-8345-80975edd50d9 None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:50.255741 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d037840a-bcba-43b0-8345-80975edd50d9 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:50.256018 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d037840a-bcba-43b0-8345-80975edd50d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:50.274375 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-d037840a-bcba-43b0-8345-80975edd50d9 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:28:50.275572 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-d037840a-bcba-43b0-8345-80975edd50d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:28:50.301836 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789d369f-41a2-45f2-8cc3-2af5f7677a00 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:50.341725 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4534e95-638f-4b14-9dc2-6c171e3f99ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:50.342603 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4534e95-638f-4b14-9dc2-6c171e3f99ca None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:50.343059 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4534e95-638f-4b14-9dc2-6c171e3f99ca None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:50.343528 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4534e95-638f-4b14-9dc2-6c171e3f99ca None admin] PATH_INFO: `/v3/services` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:50.344332 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4534e95-638f-4b14-9dc2-6c171e3f99ca None admin] RBAC: Adding query filter params (type=compute) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:50.344898 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4534e95-638f-4b14-9dc2-6c171e3f99ca None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:50.345833 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 507/1016] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:28:50 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:28:50.346635 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4534e95-638f-4b14-9dc2-6c171e3f99ca None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:50.351954 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 510/1017] 10.222.0.45 () {56 vars in 1112 bytes} [Wed Aug 24 17:28:50 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:50.357621 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d037840a-bcba-43b0-8345-80975edd50d9 None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:50.382408 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:50.461563 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d037840a-bcba-43b0-8345-80975edd50d9 None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:50.496564 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770d0b3d-d04a-4e56-a6f6-db487cc77a24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:50.497192 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770d0b3d-d04a-4e56-a6f6-db487cc77a24 None admin] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:50.497474 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770d0b3d-d04a-4e56-a6f6-db487cc77a24 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:50.497744 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770d0b3d-d04a-4e56-a6f6-db487cc77a24 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:50.498410 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770d0b3d-d04a-4e56-a6f6-db487cc77a24 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:50.499782 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770d0b3d-d04a-4e56-a6f6-db487cc77a24 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:50.528580 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 508/1018] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 17:28:50 2022] POST /identity/v3/endpoints => generated 334 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:28:50.614936 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4534e95-638f-4b14-9dc2-6c171e3f99ca None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:50.643737 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:50.722506 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4534e95-638f-4b14-9dc2-6c171e3f99ca None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:50.752451 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9a047e-53ba-413f-83fc-87fe5c11f9ae None None] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:50.752809 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9a047e-53ba-413f-83fc-87fe5c11f9ae None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:50.753093 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9a047e-53ba-413f-83fc-87fe5c11f9ae None None] PATH_INFO: `/` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:50.754441 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 509/1019] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:28:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:28:50.761216 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743a539a-334d-432d-ad54-d830b268a18b None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:50.761501 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743a539a-334d-432d-ad54-d830b268a18b None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:50.761778 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743a539a-334d-432d-ad54-d830b268a18b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:50.766505 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe22d7c2-8c6d-4c96-88be-1e86b02abee5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:50.767409 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe22d7c2-8c6d-4c96-88be-1e86b02abee5 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:50.767836 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe22d7c2-8c6d-4c96-88be-1e86b02abee5 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:50.768440 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe22d7c2-8c6d-4c96-88be-1e86b02abee5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:50.773591 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe22d7c2-8c6d-4c96-88be-1e86b02abee5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:50.775782 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe22d7c2-8c6d-4c96-88be-1e86b02abee5 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:50.777079 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 511/1020] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 17:28:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:50.782494 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe22d7c2-8c6d-4c96-88be-1e86b02abee5 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:50.784194 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-743a539a-334d-432d-ad54-d830b268a18b None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:28:50.785609 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-743a539a-334d-432d-ad54-d830b268a18b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:28:50.864344 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 510/1021] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:28:50 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:28:50.882905 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:50.896255 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe22d7c2-8c6d-4c96-88be-1e86b02abee5 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:50.929822 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1484f6f2-b2d9-481a-a701-97c3bb252dfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:50.930425 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1484f6f2-b2d9-481a-a701-97c3bb252dfa None admin] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:50.930694 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1484f6f2-b2d9-481a-a701-97c3bb252dfa None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:50.930964 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1484f6f2-b2d9-481a-a701-97c3bb252dfa None admin] PATH_INFO: `/v3/projects` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:50.931608 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1484f6f2-b2d9-481a-a701-97c3bb252dfa None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:50.933576 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1484f6f2-b2d9-481a-a701-97c3bb252dfa None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:50.968587 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 512/1022] 10.222.0.45 () {60 vars in 1141 bytes} [Wed Aug 24 17:28:50 2022] POST /identity/v3/projects => generated 311 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:28:51.130913 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743a539a-334d-432d-ad54-d830b268a18b None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:51.141279 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:51.233344 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743a539a-334d-432d-ad54-d830b268a18b None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:51.266775 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190626aa-48a9-4177-b1ec-6371de772883 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:51.267406 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190626aa-48a9-4177-b1ec-6371de772883 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:51.267684 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190626aa-48a9-4177-b1ec-6371de772883 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:51.267952 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190626aa-48a9-4177-b1ec-6371de772883 None admin] PATH_INFO: `/v3/services/image` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:51.268530 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190626aa-48a9-4177-b1ec-6371de772883 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:51.269908 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190626aa-48a9-4177-b1ec-6371de772883 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:51.278489 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application [None req-190626aa-48a9-4177-b1ec-6371de772883 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 17:28:51.278489 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:28:51.278489 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 17:28:51.278489 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 17:28:51.278489 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 17:28:51.278489 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 17:28:51.278489 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 17:28:51.278489 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:28:51.278489 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 17:28:51.278489 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Aug 24 17:28:51.278489 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 17:28:51.278489 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Aug 24 17:28:51.278489 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:28:51.278489 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:28:51.278489 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:28:51.278489 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:28:51.278489 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:28:51.279460 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:28:51.279460 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:28:51.279460 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:28:51.279460 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:28:51.279460 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:28:51.279460 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:28:51.279460 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 17:28:51.279460 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 17:28:51.279460 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 17:28:51.279460 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 17:28:51.279460 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:28:51.279460 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:28:51.279460 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:28:51.279460 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:28:51.279460 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:28:51.279460 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:28:51.280308 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:28:51.280308 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application with Lock( Aug 24 17:28:51.280308 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:28:51.280308 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:28:51.280308 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:28:51.280308 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:28:51.280308 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:28:51.280308 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:28:51.280308 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:28:51.280308 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:28:51.280308 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 17:28:51.280308 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:28:51.280308 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 17:28:51.280308 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Aug 24 17:28:51.280308 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 511/1023] 10.222.0.45 () {56 vars in 1105 bytes} [Wed Aug 24 17:28:51 2022] GET /identity/v3/services/image => generated 86 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:28:51.287275 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1484f6f2-b2d9-481a-a701-97c3bb252dfa None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:51.388120 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1484f6f2-b2d9-481a-a701-97c3bb252dfa None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:51.421879 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363fa6f5-c3f2-44ef-8c10-4164f2239cf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:51.422414 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363fa6f5-c3f2-44ef-8c10-4164f2239cf7 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:51.422692 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363fa6f5-c3f2-44ef-8c10-4164f2239cf7 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:51.422963 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363fa6f5-c3f2-44ef-8c10-4164f2239cf7 None admin] PATH_INFO: `/v3/services` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:51.423615 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363fa6f5-c3f2-44ef-8c10-4164f2239cf7 None admin] RBAC: Adding query filter params (name=image) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:51.423916 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363fa6f5-c3f2-44ef-8c10-4164f2239cf7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:51.425307 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363fa6f5-c3f2-44ef-8c10-4164f2239cf7 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:51.434775 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 513/1024] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 17:28:51 2022] GET /identity/v3/services?name=image => generated 122 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:28:51.436151 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190626aa-48a9-4177-b1ec-6371de772883 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:51.509260 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:51.540841 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190626aa-48a9-4177-b1ec-6371de772883 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:51.574340 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48795575-5a83-4b0c-9784-3836a84b6951 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:51.575107 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48795575-5a83-4b0c-9784-3836a84b6951 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:51.575392 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48795575-5a83-4b0c-9784-3836a84b6951 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:51.575664 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48795575-5a83-4b0c-9784-3836a84b6951 None admin] PATH_INFO: `/v3/services` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:51.576289 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48795575-5a83-4b0c-9784-3836a84b6951 None admin] RBAC: Adding query filter params (type=image) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:51.576615 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48795575-5a83-4b0c-9784-3836a84b6951 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:51.577973 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48795575-5a83-4b0c-9784-3836a84b6951 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:51.588878 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 512/1025] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 17:28:51 2022] GET /identity/v3/services?type=image => generated 350 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:51.590853 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363fa6f5-c3f2-44ef-8c10-4164f2239cf7 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:51.595589 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:51.696168 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363fa6f5-c3f2-44ef-8c10-4164f2239cf7 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:51.730697 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668d11a3-593a-486d-9d40-46260db95fec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:51.731306 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668d11a3-593a-486d-9d40-46260db95fec None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:51.731840 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668d11a3-593a-486d-9d40-46260db95fec None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:51.732111 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668d11a3-593a-486d-9d40-46260db95fec None admin] PATH_INFO: `/v3/endpoints` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:51.732829 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668d11a3-593a-486d-9d40-46260db95fec None admin] RBAC: Adding query filter params (interface=public, service_id=24d910d62ee740bdba150ba994c0fd60, region_id=RegionOne) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:51.733137 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668d11a3-593a-486d-9d40-46260db95fec None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:51.734498 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668d11a3-593a-486d-9d40-46260db95fec None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:51.743979 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 514/1026] 10.222.0.45 () {56 vars in 1251 bytes} [Wed Aug 24 17:28:51 2022] GET /identity/v3/endpoints?service_id=24d910d62ee740bdba150ba994c0fd60&interface=public®ion_id=RegionOne => generated 194 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:51.745252 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48795575-5a83-4b0c-9784-3836a84b6951 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:51.854025 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48795575-5a83-4b0c-9784-3836a84b6951 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:51.887354 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1415b248-0f7f-46a4-8e3a-61546a4b1185 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:51.887966 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1415b248-0f7f-46a4-8e3a-61546a4b1185 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:51.888242 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1415b248-0f7f-46a4-8e3a-61546a4b1185 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:51.888512 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1415b248-0f7f-46a4-8e3a-61546a4b1185 None admin] PATH_INFO: `/v3/services` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:51.889135 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1415b248-0f7f-46a4-8e3a-61546a4b1185 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:51.890481 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1415b248-0f7f-46a4-8e3a-61546a4b1185 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:51.892189 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:51.898435 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 513/1027] 10.222.0.45 () {56 vars in 1087 bytes} [Wed Aug 24 17:28:51 2022] GET /identity/v3/services => generated 1957 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 17:28:52.532026 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e65711d-2b62-4cd5-8b85-2d81c2202e06 None None] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:52.532365 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e65711d-2b62-4cd5-8b85-2d81c2202e06 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:52.532610 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e65711d-2b62-4cd5-8b85-2d81c2202e06 None None] PATH_INFO: `/` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:52.533992 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 515/1028] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:28:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:28:52.539332 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66d2a49-6c3f-46f9-bbfa-176b20c10069 None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:52.539617 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66d2a49-6c3f-46f9-bbfa-176b20c10069 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:52.539892 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66d2a49-6c3f-46f9-bbfa-176b20c10069 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:52.558555 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-d66d2a49-6c3f-46f9-bbfa-176b20c10069 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:28:52.559728 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-d66d2a49-6c3f-46f9-bbfa-176b20c10069 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:28:52.627511 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 514/1029] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:28:52 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:28:52.893297 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e65711d-2b62-4cd5-8b85-2d81c2202e06 None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:52.925545 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761f7a3e-3cfe-4781-a4f0-944b46cd7e8e None None] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:52.925839 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761f7a3e-3cfe-4781-a4f0-944b46cd7e8e None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:52.926123 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761f7a3e-3cfe-4781-a4f0-944b46cd7e8e None None] PATH_INFO: `/` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:52.927464 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 515/1030] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:28:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:28:52.932303 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0a33b2-288c-4e0f-9019-b3df63b61cf7 None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:52.932610 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0a33b2-288c-4e0f-9019-b3df63b61cf7 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:52.932890 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0a33b2-288c-4e0f-9019-b3df63b61cf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:52.951590 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-dc0a33b2-288c-4e0f-9019-b3df63b61cf7 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:28:52.952822 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-dc0a33b2-288c-4e0f-9019-b3df63b61cf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:28:52.966592 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:53.005638 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e65711d-2b62-4cd5-8b85-2d81c2202e06 None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:53.024596 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 516/1031] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:28:52 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 17:28:53.043001 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be593b1-90fa-4d9b-babc-89870e8cc929 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:53.043628 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be593b1-90fa-4d9b-babc-89870e8cc929 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:53.043903 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be593b1-90fa-4d9b-babc-89870e8cc929 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:53.044169 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be593b1-90fa-4d9b-babc-89870e8cc929 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:53.054626 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be593b1-90fa-4d9b-babc-89870e8cc929 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:28:53.059753 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application [None req-1be593b1-90fa-4d9b-babc-89870e8cc929 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 17:28:53.059753 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:28:53.059753 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:28:53.059753 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:28:53.059753 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:28:53.059753 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:28:53.059753 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:28:53.059753 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:28:53.059753 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:28:53.059753 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:28:53.059753 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:28:53.059753 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:28:53.059753 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:28:53.059753 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:28:53.059753 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:28:53.059753 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:28:53.060689 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:28:53.060689 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:28:53.060689 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:28:53.060689 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:28:53.060689 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:28:53.060689 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:28:53.060689 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:28:53.060689 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:28:53.060689 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:28:53.060689 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:28:53.060689 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:28:53.060689 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application with Lock( Aug 24 17:28:53.060689 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:28:53.060689 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:28:53.060689 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:28:53.060689 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:28:53.065555 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:28:53.065555 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:28:53.065555 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:28:53.065555 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:28:53.065555 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:28:53.065555 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:28:53.065555 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:28:53.065555 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:28:53.065555 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:28:53.065555 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:28:53.065555 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 17:28:53.065555 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Aug 24 17:28:53.065555 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 516/1032] 10.222.0.45 () {56 vars in 1120 bytes} [Wed Aug 24 17:28:52 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:28:53.067432 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0a33b2-288c-4e0f-9019-b3df63b61cf7 None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:53.126733 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:53.168495 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0a33b2-288c-4e0f-9019-b3df63b61cf7 None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:53.202023 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f027778-5a2a-47cb-aaab-4ac2f491d25f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:53.202627 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f027778-5a2a-47cb-aaab-4ac2f491d25f None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:53.202906 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f027778-5a2a-47cb-aaab-4ac2f491d25f None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:53.203177 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f027778-5a2a-47cb-aaab-4ac2f491d25f None admin] PATH_INFO: `/v3/roles` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:53.203836 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f027778-5a2a-47cb-aaab-4ac2f491d25f None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:53.204138 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f027778-5a2a-47cb-aaab-4ac2f491d25f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:53.205525 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f027778-5a2a-47cb-aaab-4ac2f491d25f None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:53.217992 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 517/1033] 10.222.0.45 () {56 vars in 1115 bytes} [Wed Aug 24 17:28:53 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:53.219453 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be593b1-90fa-4d9b-babc-89870e8cc929 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:53.296836 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f027778-5a2a-47cb-aaab-4ac2f491d25f None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:53.319406 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be593b1-90fa-4d9b-babc-89870e8cc929 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:53.346830 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:53.358686 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b18abb-f817-4e22-a810-2deafaad7369 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:53.359621 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b18abb-f817-4e22-a810-2deafaad7369 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:53.360039 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b18abb-f817-4e22-a810-2deafaad7369 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:53.360441 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b18abb-f817-4e22-a810-2deafaad7369 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:53.373023 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b18abb-f817-4e22-a810-2deafaad7369 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:53.374462 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b18abb-f817-4e22-a810-2deafaad7369 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:53.389163 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application [None req-74b18abb-f817-4e22-a810-2deafaad7369 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:28:53.389163 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:28:53.389163 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:28:53.389163 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:28:53.389163 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:28:53.389163 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:28:53.389163 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:28:53.389163 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:28:53.389163 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:28:53.389163 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:28:53.389163 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:28:53.389163 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:28:53.389163 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:28:53.389163 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:28:53.389163 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:28:53.389163 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:28:53.390121 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:28:53.390121 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:28:53.390121 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:28:53.390121 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:28:53.390121 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:28:53.390121 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:28:53.390121 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:28:53.390121 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application with Lock( Aug 24 17:28:53.390121 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:28:53.390121 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:28:53.390121 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:28:53.390121 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:28:53.390121 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:28:53.390121 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:28:53.390121 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:28:53.390121 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:28:53.390121 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:28:53.391608 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:28:53.391608 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:28:53.391608 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Aug 24 17:28:53.391608 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 517/1034] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 17:28:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 17:28:53.395955 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b18abb-f817-4e22-a810-2deafaad7369 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:53.401532 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f027778-5a2a-47cb-aaab-4ac2f491d25f None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:53.425224 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:53.437885 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b85bd37-ed9c-42a4-8ff1-ec40e5a45aa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:53.438522 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b85bd37-ed9c-42a4-8ff1-ec40e5a45aa1 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:53.438801 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b85bd37-ed9c-42a4-8ff1-ec40e5a45aa1 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:53.439074 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b85bd37-ed9c-42a4-8ff1-ec40e5a45aa1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:53.440189 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b85bd37-ed9c-42a4-8ff1-ec40e5a45aa1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:53.441661 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b85bd37-ed9c-42a4-8ff1-ec40e5a45aa1 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:53.443511 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 518/1035] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 17:28:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:28:53.448811 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b85bd37-ed9c-42a4-8ff1-ec40e5a45aa1 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:53.511876 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b18abb-f817-4e22-a810-2deafaad7369 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:53.546347 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ed8d69-664a-41b1-b74d-e6a26ff638e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:53.546960 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ed8d69-664a-41b1-b74d-e6a26ff638e8 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:53.547241 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ed8d69-664a-41b1-b74d-e6a26ff638e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:53.547507 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ed8d69-664a-41b1-b74d-e6a26ff638e8 None admin] PATH_INFO: `/v3/domains` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:53.548157 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ed8d69-664a-41b1-b74d-e6a26ff638e8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:53.548516 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ed8d69-664a-41b1-b74d-e6a26ff638e8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:53.549913 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ed8d69-664a-41b1-b74d-e6a26ff638e8 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:53.558138 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b85bd37-ed9c-42a4-8ff1-ec40e5a45aa1 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:53.566861 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 518/1036] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 24 17:28:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:28:53.567879 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ed8d69-664a-41b1-b74d-e6a26ff638e8 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:53.594118 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0c12ca-cabe-4ef2-aef3-2a35bdb30b33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:53.594652 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0c12ca-cabe-4ef2-aef3-2a35bdb30b33 None admin] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:53.594925 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0c12ca-cabe-4ef2-aef3-2a35bdb30b33 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:53.595191 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0c12ca-cabe-4ef2-aef3-2a35bdb30b33 None admin] PATH_INFO: `/v3/users` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:53.595832 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0c12ca-cabe-4ef2-aef3-2a35bdb30b33 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:53.597282 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0c12ca-cabe-4ef2-aef3-2a35bdb30b33 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:53.618253 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:53.624606 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 519/1037] 10.222.0.45 () {60 vars in 1132 bytes} [Wed Aug 24 17:28:53 2022] POST /identity/v3/users => generated 285 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:28:53.688826 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ed8d69-664a-41b1-b74d-e6a26ff638e8 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:53.725544 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6556845e-d63b-4121-b16e-dca3d9597db8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:53.726195 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6556845e-d63b-4121-b16e-dca3d9597db8 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:53.726478 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6556845e-d63b-4121-b16e-dca3d9597db8 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:53.726750 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6556845e-d63b-4121-b16e-dca3d9597db8 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:53.735975 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6556845e-d63b-4121-b16e-dca3d9597db8 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:53.737496 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6556845e-d63b-4121-b16e-dca3d9597db8 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:53.750010 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application [None req-6556845e-d63b-4121-b16e-dca3d9597db8 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 17:28:53.750010 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:28:53.750010 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:28:53.750010 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:28:53.750010 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:28:53.750010 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:28:53.750010 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:28:53.750010 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:28:53.750010 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:28:53.750010 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:28:53.750010 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:28:53.750010 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:28:53.750010 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 17:28:53.750010 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 17:28:53.750010 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 17:28:53.750010 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 17:28:53.751069 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:28:53.751069 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:28:53.751069 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 17:28:53.751069 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:28:53.751069 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 17:28:53.751069 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:28:53.751069 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:28:53.751069 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:28:53.751069 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:28:53.751069 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:28:53.751069 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:28:53.751069 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application with Lock( Aug 24 17:28:53.751069 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:28:53.751069 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:28:53.751069 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:28:53.751069 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:28:53.751069 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:28:53.751069 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:28:53.757191 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:28:53.757191 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:28:53.757191 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 17:28:53.757191 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 17:28:53.757191 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 17:28:53.757191 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 17:28:53.757191 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 17:28:53.757191 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 17:28:53.757191 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 17:28:53.757191 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Aug 24 17:28:53.757191 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 519/1038] 10.222.0.45 () {56 vars in 1093 bytes} [Wed Aug 24 17:28:53 2022] GET /identity/v3/users/nova => generated 82 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:28:53.759136 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0c12ca-cabe-4ef2-aef3-2a35bdb30b33 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:53.806244 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984a7b32-3975-4791-9ac7-31bd68550fea None None] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:53.806535 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984a7b32-3975-4791-9ac7-31bd68550fea None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:53.806812 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984a7b32-3975-4791-9ac7-31bd68550fea None None] PATH_INFO: `/` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:53.808181 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 520/1039] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:28:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 17:28:53.812607 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:53.813426 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d96d6d0-57df-45e6-855b-e3e86d61365c None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:53.813711 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d96d6d0-57df-45e6-855b-e3e86d61365c None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:53.814019 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d96d6d0-57df-45e6-855b-e3e86d61365c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:53.833363 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-9d96d6d0-57df-45e6-855b-e3e86d61365c None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:28:53.834579 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-9d96d6d0-57df-45e6-855b-e3e86d61365c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:28:53.878396 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0c12ca-cabe-4ef2-aef3-2a35bdb30b33 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:53.907755 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 521/1040] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:28:53 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 17:28:53.914535 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20957cc6-39d2-4cbd-8747-5b879b72c10d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:53.915148 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20957cc6-39d2-4cbd-8747-5b879b72c10d None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:53.915432 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20957cc6-39d2-4cbd-8747-5b879b72c10d None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:53.915708 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20957cc6-39d2-4cbd-8747-5b879b72c10d None admin] PATH_INFO: `/v3/users` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:53.916495 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20957cc6-39d2-4cbd-8747-5b879b72c10d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:53.917336 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20957cc6-39d2-4cbd-8747-5b879b72c10d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:53.918794 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20957cc6-39d2-4cbd-8747-5b879b72c10d None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:53.936714 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 520/1041] 10.222.0.45 () {56 vars in 1133 bytes} [Wed Aug 24 17:28:53 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:53.942163 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d96d6d0-57df-45e6-855b-e3e86d61365c None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:53.972433 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:54.043502 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d96d6d0-57df-45e6-855b-e3e86d61365c None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:54.076808 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75f0e23-f53d-4468-9006-491a05b7f03a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:54.077416 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75f0e23-f53d-4468-9006-491a05b7f03a None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:54.077691 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75f0e23-f53d-4468-9006-491a05b7f03a None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:54.077960 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75f0e23-f53d-4468-9006-491a05b7f03a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:54.144862 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75f0e23-f53d-4468-9006-491a05b7f03a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:54.146333 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75f0e23-f53d-4468-9006-491a05b7f03a None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:54.174019 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20957cc6-39d2-4cbd-8747-5b879b72c10d None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:54.181312 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:54.255764 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 522/1042] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Aug 24 17:28:53 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 17:28:54.261886 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75f0e23-f53d-4468-9006-491a05b7f03a None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:54.282534 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20957cc6-39d2-4cbd-8747-5b879b72c10d None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:54.285008 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:54.320590 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87849db9-4411-4189-8a86-e1b728ad1782 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:54.321225 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87849db9-4411-4189-8a86-e1b728ad1782 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:54.321507 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87849db9-4411-4189-8a86-e1b728ad1782 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:54.321782 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87849db9-4411-4189-8a86-e1b728ad1782 None admin] PATH_INFO: `/v3/services/image` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:54.322371 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87849db9-4411-4189-8a86-e1b728ad1782 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:54.324491 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87849db9-4411-4189-8a86-e1b728ad1782 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:54.334071 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application [None req-87849db9-4411-4189-8a86-e1b728ad1782 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 17:28:54.334071 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:28:54.334071 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 17:28:54.334071 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 17:28:54.334071 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 17:28:54.334071 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 17:28:54.334071 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 17:28:54.334071 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:28:54.334071 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 17:28:54.334071 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Aug 24 17:28:54.334071 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 17:28:54.334071 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Aug 24 17:28:54.334071 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:28:54.334071 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:28:54.334071 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:28:54.334071 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:28:54.334071 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:28:54.335078 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:28:54.335078 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:28:54.335078 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:28:54.335078 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:28:54.335078 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:28:54.335078 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:28:54.335078 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 17:28:54.335078 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 17:28:54.335078 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 17:28:54.335078 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 17:28:54.335078 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:28:54.335078 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:28:54.335078 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:28:54.335078 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:28:54.335078 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:28:54.335078 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:28:54.335954 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:28:54.335954 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application with Lock( Aug 24 17:28:54.335954 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:28:54.335954 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:28:54.335954 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:28:54.335954 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:28:54.335954 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:28:54.335954 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:28:54.335954 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:28:54.335954 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:28:54.335954 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 17:28:54.335954 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:28:54.335954 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 17:28:54.335954 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Aug 24 17:28:54.335954 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 521/1043] 10.222.0.45 () {56 vars in 1105 bytes} [Wed Aug 24 17:28:54 2022] GET /identity/v3/services/image => generated 86 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 17:28:54.341725 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87849db9-4411-4189-8a86-e1b728ad1782 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:54.369412 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75f0e23-f53d-4468-9006-491a05b7f03a None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:54.398629 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:54.406748 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfd6936-8ff7-4edd-900c-42ced3c1c672 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:54.407380 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfd6936-8ff7-4edd-900c-42ced3c1c672 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:54.407657 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfd6936-8ff7-4edd-900c-42ced3c1c672 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:54.407928 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfd6936-8ff7-4edd-900c-42ced3c1c672 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:54.418287 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfd6936-8ff7-4edd-900c-42ced3c1c672 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:54.419741 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfd6936-8ff7-4edd-900c-42ced3c1c672 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:54.441160 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application [None req-1bfd6936-8ff7-4edd-900c-42ced3c1c672 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:28:54.441160 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:28:54.441160 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:28:54.441160 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:28:54.441160 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:28:54.441160 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:28:54.441160 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:28:54.441160 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:28:54.441160 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:28:54.441160 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:28:54.441160 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:28:54.441160 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:28:54.441160 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:28:54.441160 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:28:54.441160 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:28:54.441160 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:28:54.442140 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:28:54.442140 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:28:54.442140 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:28:54.442140 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:28:54.442140 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:28:54.442140 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:28:54.442140 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:28:54.442140 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application with Lock( Aug 24 17:28:54.442140 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:28:54.442140 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:28:54.442140 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:28:54.442140 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:28:54.442140 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:28:54.442140 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:28:54.442140 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:28:54.442140 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:28:54.442140 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:28:54.443587 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:28:54.443587 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:28:54.443587 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Aug 24 17:28:54.443587 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 523/1044] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 17:28:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:28:54.448468 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfd6936-8ff7-4edd-900c-42ced3c1c672 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:54.449681 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87849db9-4411-4189-8a86-e1b728ad1782 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:54.485511 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263a0a22-bf06-4388-ba67-d97ac4f12104 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:54.486112 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263a0a22-bf06-4388-ba67-d97ac4f12104 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:54.486388 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263a0a22-bf06-4388-ba67-d97ac4f12104 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:54.486657 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263a0a22-bf06-4388-ba67-d97ac4f12104 None admin] PATH_INFO: `/v3/services` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:54.487278 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263a0a22-bf06-4388-ba67-d97ac4f12104 None admin] RBAC: Adding query filter params (name=image) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:54.487576 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263a0a22-bf06-4388-ba67-d97ac4f12104 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:54.488959 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263a0a22-bf06-4388-ba67-d97ac4f12104 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:54.496589 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 522/1045] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 17:28:54 2022] GET /identity/v3/services?name=image => generated 122 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:28:54.500216 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263a0a22-bf06-4388-ba67-d97ac4f12104 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:54.534169 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:54.554846 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfd6936-8ff7-4edd-900c-42ced3c1c672 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:54.589789 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b34c055-701c-4076-a615-a2a9ff2e5472 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:54.590403 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b34c055-701c-4076-a615-a2a9ff2e5472 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:54.590683 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b34c055-701c-4076-a615-a2a9ff2e5472 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:54.590964 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b34c055-701c-4076-a615-a2a9ff2e5472 None admin] PATH_INFO: `/v3/domains` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:54.591604 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b34c055-701c-4076-a615-a2a9ff2e5472 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:54.591909 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b34c055-701c-4076-a615-a2a9ff2e5472 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:54.593301 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b34c055-701c-4076-a615-a2a9ff2e5472 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:54.609616 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 524/1046] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 24 17:28:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:28:54.609798 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263a0a22-bf06-4388-ba67-d97ac4f12104 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:54.611728 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b34c055-701c-4076-a615-a2a9ff2e5472 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:54.651076 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b978b2-1bb3-4b3c-abf0-eef63462ef17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:54.651615 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b978b2-1bb3-4b3c-abf0-eef63462ef17 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:54.651895 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b978b2-1bb3-4b3c-abf0-eef63462ef17 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:54.652167 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b978b2-1bb3-4b3c-abf0-eef63462ef17 None admin] PATH_INFO: `/v3/services` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:54.652828 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b978b2-1bb3-4b3c-abf0-eef63462ef17 None admin] RBAC: Adding query filter params (type=image) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:54.653190 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b978b2-1bb3-4b3c-abf0-eef63462ef17 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:54.655007 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b978b2-1bb3-4b3c-abf0-eef63462ef17 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:54.657426 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:54.667338 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 523/1047] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 17:28:54 2022] GET /identity/v3/services?type=image => generated 350 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:28:54.668772 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b978b2-1bb3-4b3c-abf0-eef63462ef17 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:54.715589 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b34c055-701c-4076-a615-a2a9ff2e5472 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:54.752442 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735e74bb-f570-4a04-b667-69ebbb43858f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:54.753126 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735e74bb-f570-4a04-b667-69ebbb43858f None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:54.753406 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735e74bb-f570-4a04-b667-69ebbb43858f None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:54.753678 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735e74bb-f570-4a04-b667-69ebbb43858f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:54.761850 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735e74bb-f570-4a04-b667-69ebbb43858f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:54.763344 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735e74bb-f570-4a04-b667-69ebbb43858f None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:54.777997 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application [None req-735e74bb-f570-4a04-b667-69ebbb43858f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:28:54.777997 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:28:54.777997 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:28:54.777997 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:28:54.777997 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:28:54.777997 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:28:54.777997 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:28:54.777997 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:28:54.777997 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:28:54.777997 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:28:54.777997 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:28:54.777997 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:28:54.777997 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 17:28:54.777997 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 17:28:54.777997 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 17:28:54.778942 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 17:28:54.778942 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:28:54.778942 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:28:54.778942 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:28:54.778942 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:28:54.778942 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:28:54.778942 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:28:54.778942 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:28:54.778942 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application with Lock( Aug 24 17:28:54.778942 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:28:54.778942 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:28:54.778942 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:28:54.778942 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:28:54.778942 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:28:54.778942 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:28:54.778942 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:28:54.778942 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:28:54.779827 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 17:28:54.779827 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 17:28:54.779827 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 17:28:54.779827 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 17:28:54.779827 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 17:28:54.779827 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 17:28:54.779827 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:28:54.779827 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Aug 24 17:28:54.779827 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 525/1048] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Aug 24 17:28:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 17:28:54.781874 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:54.785428 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735e74bb-f570-4a04-b667-69ebbb43858f None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:54.788347 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b978b2-1bb3-4b3c-abf0-eef63462ef17 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:54.828091 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f21f65-d829-41c8-8435-d0ef6a44849c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:54.828754 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f21f65-d829-41c8-8435-d0ef6a44849c None admin] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:54.829033 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f21f65-d829-41c8-8435-d0ef6a44849c None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:54.829417 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f21f65-d829-41c8-8435-d0ef6a44849c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:54.830074 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f21f65-d829-41c8-8435-d0ef6a44849c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:54.831547 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f21f65-d829-41c8-8435-d0ef6a44849c None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:54.857364 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 524/1049] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 17:28:54 2022] POST /identity/v3/endpoints => generated 327 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:28:54.904372 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735e74bb-f570-4a04-b667-69ebbb43858f None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:54.906126 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:54.949329 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210a9a14-195c-4e65-86ff-134c4d58eb44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:54.950026 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210a9a14-195c-4e65-86ff-134c4d58eb44 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:54.950277 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210a9a14-195c-4e65-86ff-134c4d58eb44 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:54.950478 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210a9a14-195c-4e65-86ff-134c4d58eb44 None admin] PATH_INFO: `/v3/projects` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:54.951174 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210a9a14-195c-4e65-86ff-134c4d58eb44 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:54.951478 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210a9a14-195c-4e65-86ff-134c4d58eb44 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:54.952963 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210a9a14-195c-4e65-86ff-134c4d58eb44 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:54.966320 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 526/1050] 10.222.0.45 () {56 vars in 1148 bytes} [Wed Aug 24 17:28:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:54.972476 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f21f65-d829-41c8-8435-d0ef6a44849c None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:55.082617 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f21f65-d829-41c8-8435-d0ef6a44849c None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:55.116293 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2624359-b94d-47b9-b01c-07077e70c128 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:55.116927 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2624359-b94d-47b9-b01c-07077e70c128 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:55.117204 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2624359-b94d-47b9-b01c-07077e70c128 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:55.117473 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2624359-b94d-47b9-b01c-07077e70c128 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:55.118198 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2624359-b94d-47b9-b01c-07077e70c128 None admin] RBAC: Adding query filter params (role.id=d6d7b68e2dee4176bb8712ffbac4808f, scope.project.id=2ad8a32bd9d2434097bd08b111914ca1, user.id=053b7ce6d6f649ed91a26e434dab805e) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:55.118504 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2624359-b94d-47b9-b01c-07077e70c128 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:55.120487 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2624359-b94d-47b9-b01c-07077e70c128 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:55.126127 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 525/1051] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Aug 24 17:28:54 2022] GET /identity/v3/role_assignments?user.id=053b7ce6d6f649ed91a26e434dab805e&scope.project.id=2ad8a32bd9d2434097bd08b111914ca1&role.id=d6d7b68e2dee4176bb8712ffbac4808f => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:55.611794 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad541b15-7221-44d7-be90-08b2a8f7178a None None] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:55.612145 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad541b15-7221-44d7-be90-08b2a8f7178a None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:55.612361 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad541b15-7221-44d7-be90-08b2a8f7178a None None] PATH_INFO: `/` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:55.613765 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 527/1052] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:28:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:28:55.618919 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20e472a-ce20-4712-bd2a-52a936778da6 None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:55.619401 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20e472a-ce20-4712-bd2a-52a936778da6 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:55.619478 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20e472a-ce20-4712-bd2a-52a936778da6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:55.637713 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-a20e472a-ce20-4712-bd2a-52a936778da6 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:28:55.638894 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-a20e472a-ce20-4712-bd2a-52a936778da6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:28:55.649526 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:55.706983 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 526/1053] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:28:55 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:28:55.976590 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad541b15-7221-44d7-be90-08b2a8f7178a None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:56.081053 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad541b15-7221-44d7-be90-08b2a8f7178a None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:56.115538 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e28be69-c8eb-4828-b920-68128165d713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:56.116172 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e28be69-c8eb-4828-b920-68128165d713 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:56.116457 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e28be69-c8eb-4828-b920-68128165d713 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:56.116761 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e28be69-c8eb-4828-b920-68128165d713 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:56.127355 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e28be69-c8eb-4828-b920-68128165d713 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:28:56.132016 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application [None req-8e28be69-c8eb-4828-b920-68128165d713 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:28:56.132016 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:28:56.132016 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:28:56.132016 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:28:56.132016 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:28:56.132016 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:28:56.132016 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:28:56.132016 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:28:56.132016 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:28:56.132016 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:28:56.132016 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:28:56.132016 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:28:56.132016 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:28:56.132016 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:28:56.132016 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:28:56.132016 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:28:56.133118 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:28:56.133118 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:28:56.133118 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:28:56.133118 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:28:56.133118 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:28:56.133118 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:28:56.133118 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:28:56.133118 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:28:56.133118 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:28:56.133118 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:28:56.133118 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:28:56.133118 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application with Lock( Aug 24 17:28:56.133118 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:28:56.133118 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:28:56.133118 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:28:56.133118 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:28:56.133998 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:28:56.133998 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:28:56.133998 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:28:56.133998 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:28:56.133998 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:28:56.133998 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:28:56.133998 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:28:56.133998 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:28:56.133998 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:28:56.133998 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:28:56.133998 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:28:56.133998 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Aug 24 17:28:56.136870 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 528/1054] 10.222.0.45 () {56 vars in 1096 bytes} [Wed Aug 24 17:28:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:28:56.139601 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20e472a-ce20-4712-bd2a-52a936778da6 None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:56.244045 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20e472a-ce20-4712-bd2a-52a936778da6 None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:56.278755 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e2d1d4-eec8-4622-a98a-283583ba296c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:56.279371 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e2d1d4-eec8-4622-a98a-283583ba296c None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:56.279652 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e2d1d4-eec8-4622-a98a-283583ba296c None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:56.279923 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e2d1d4-eec8-4622-a98a-283583ba296c None admin] PATH_INFO: `/v3/roles` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:56.280607 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e2d1d4-eec8-4622-a98a-283583ba296c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:56.280914 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e2d1d4-eec8-4622-a98a-283583ba296c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:56.282273 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e2d1d4-eec8-4622-a98a-283583ba296c None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:56.295898 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 527/1055] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Aug 24 17:28:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:56.297102 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e28be69-c8eb-4828-b920-68128165d713 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:56.317312 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:56.399222 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e28be69-c8eb-4828-b920-68128165d713 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:56.433848 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f4ef5f-d55e-4803-a30a-894e2f7d0a75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:56.434486 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f4ef5f-d55e-4803-a30a-894e2f7d0a75 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:56.434767 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f4ef5f-d55e-4803-a30a-894e2f7d0a75 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:56.435045 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f4ef5f-d55e-4803-a30a-894e2f7d0a75 None admin] PATH_INFO: `/v3/users/16f4741af0d3432db93cb693fcbd7f52` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:56.448317 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f4ef5f-d55e-4803-a30a-894e2f7d0a75 None admin] RBAC: Authorizing `identity:get_user(user_id=16f4741af0d3432db93cb693fcbd7f52)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:56.449802 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f4ef5f-d55e-4803-a30a-894e2f7d0a75 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:56.456518 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 529/1056] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Aug 24 17:28:56 2022] GET /identity/v3/users/16f4741af0d3432db93cb693fcbd7f52 => generated 285 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:28:56.457527 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e2d1d4-eec8-4622-a98a-283583ba296c None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:56.471304 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:56.572158 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e2d1d4-eec8-4622-a98a-283583ba296c None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:56.606568 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107b1dc7-407f-4d6f-80ed-a9d401ee53dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:56.607433 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107b1dc7-407f-4d6f-80ed-a9d401ee53dc None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:56.607713 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107b1dc7-407f-4d6f-80ed-a9d401ee53dc None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:56.607988 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107b1dc7-407f-4d6f-80ed-a9d401ee53dc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:56.677843 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107b1dc7-407f-4d6f-80ed-a9d401ee53dc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:56.679305 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107b1dc7-407f-4d6f-80ed-a9d401ee53dc None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:56.689536 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:56.786647 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 528/1057] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Aug 24 17:28:56 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 17:28:56.791246 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce38b6f-c5ec-4e7d-b865-ff9e8f865574 None None] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:56.792236 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce38b6f-c5ec-4e7d-b865-ff9e8f865574 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:56.792517 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce38b6f-c5ec-4e7d-b865-ff9e8f865574 None None] PATH_INFO: `/` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:56.793290 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107b1dc7-407f-4d6f-80ed-a9d401ee53dc None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:56.797372 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 530/1058] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:28:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:28:56.799314 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f7ac00-189e-499e-848b-b04a96f4e314 None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:56.799435 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f7ac00-189e-499e-848b-b04a96f4e314 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:56.799677 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f7ac00-189e-499e-848b-b04a96f4e314 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:56.818175 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-28f7ac00-189e-499e-848b-b04a96f4e314 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:28:56.819353 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-28f7ac00-189e-499e-848b-b04a96f4e314 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:28:56.887832 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 531/1059] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:28:56 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 17:28:56.900225 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107b1dc7-407f-4d6f-80ed-a9d401ee53dc None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:56.910062 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:56.936067 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d496350-b1ff-4ab3-b3b8-03c10fac0d44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:56.936734 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d496350-b1ff-4ab3-b3b8-03c10fac0d44 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:56.937018 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d496350-b1ff-4ab3-b3b8-03c10fac0d44 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:56.937294 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d496350-b1ff-4ab3-b3b8-03c10fac0d44 None admin] PATH_INFO: `/v3/projects/13b5590b4c0c4eeb84f953f2dcea1bbe` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:56.938403 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d496350-b1ff-4ab3-b3b8-03c10fac0d44 None admin] RBAC: Authorizing `identity:get_project(project_id=13b5590b4c0c4eeb84f953f2dcea1bbe)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:56.939892 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d496350-b1ff-4ab3-b3b8-03c10fac0d44 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:56.946471 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 529/1060] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Aug 24 17:28:56 2022] GET /identity/v3/projects/13b5590b4c0c4eeb84f953f2dcea1bbe => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:56.947664 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f7ac00-189e-499e-848b-b04a96f4e314 None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:56.956263 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:56.998177 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34336165-bc56-4b0e-b468-ba26db93f8f4 None None] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:56.998462 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34336165-bc56-4b0e-b468-ba26db93f8f4 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:56.998737 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34336165-bc56-4b0e-b468-ba26db93f8f4 None None] PATH_INFO: `/` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:57.000180 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 530/1061] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:28:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 17:28:57.005078 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e116f86-06ff-4b78-be08-5f5122236b04 None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:57.005361 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e116f86-06ff-4b78-be08-5f5122236b04 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:57.005632 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e116f86-06ff-4b78-be08-5f5122236b04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:57.024385 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-2e116f86-06ff-4b78-be08-5f5122236b04 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:28:57.025601 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-2e116f86-06ff-4b78-be08-5f5122236b04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:28:57.056373 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f7ac00-189e-499e-848b-b04a96f4e314 None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:57.093953 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35903f7-a406-4634-b033-7a2c09cdb876 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:57.094907 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35903f7-a406-4634-b033-7a2c09cdb876 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:57.095134 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35903f7-a406-4634-b033-7a2c09cdb876 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:57.095417 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35903f7-a406-4634-b033-7a2c09cdb876 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:57.096142 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35903f7-a406-4634-b033-7a2c09cdb876 None admin] RBAC: Adding query filter params (role.id=e411ad9eb96e4e8d85b59b5b700ada7b, scope.project.id=13b5590b4c0c4eeb84f953f2dcea1bbe, user.id=16f4741af0d3432db93cb693fcbd7f52) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:57.096453 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35903f7-a406-4634-b033-7a2c09cdb876 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:57.100584 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 531/1062] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:28:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 17:28:57.100722 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35903f7-a406-4634-b033-7a2c09cdb876 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:57.103727 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 532/1063] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Aug 24 17:28:56 2022] GET /identity/v3/role_assignments?user.id=16f4741af0d3432db93cb693fcbd7f52&scope.project.id=13b5590b4c0c4eeb84f953f2dcea1bbe&role.id=e411ad9eb96e4e8d85b59b5b700ada7b => generated 259 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:28:57.157132 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e116f86-06ff-4b78-be08-5f5122236b04 None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:57.268223 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e116f86-06ff-4b78-be08-5f5122236b04 None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:57.276526 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:57.304393 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b859476d-8e90-41c3-a25d-47b0aa6b2e41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:57.305018 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b859476d-8e90-41c3-a25d-47b0aa6b2e41 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:57.305294 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b859476d-8e90-41c3-a25d-47b0aa6b2e41 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:57.305563 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b859476d-8e90-41c3-a25d-47b0aa6b2e41 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:57.365853 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35903f7-a406-4634-b033-7a2c09cdb876 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:57.373478 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b859476d-8e90-41c3-a25d-47b0aa6b2e41 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:57.374932 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b859476d-8e90-41c3-a25d-47b0aa6b2e41 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:57.443718 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:57.476394 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35903f7-a406-4634-b033-7a2c09cdb876 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:57.486484 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 532/1064] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Aug 24 17:28:57 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 17:28:57.492089 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b859476d-8e90-41c3-a25d-47b0aa6b2e41 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:57.501148 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:57.519966 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26d9db4-f7ec-4594-b63a-8e410323a380 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:57.520624 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26d9db4-f7ec-4594-b63a-8e410323a380 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:57.520899 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26d9db4-f7ec-4594-b63a-8e410323a380 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:57.521386 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26d9db4-f7ec-4594-b63a-8e410323a380 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:57.531986 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26d9db4-f7ec-4594-b63a-8e410323a380 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:28:57.536833 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application [None req-c26d9db4-f7ec-4594-b63a-8e410323a380 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 17:28:57.536833 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:28:57.536833 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:28:57.536833 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:28:57.536833 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:28:57.536833 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:28:57.536833 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:28:57.536833 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:28:57.536833 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:28:57.536833 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:28:57.536833 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:28:57.536833 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:28:57.536833 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:28:57.536833 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:28:57.536833 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:28:57.536833 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:28:57.537852 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:28:57.537852 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:28:57.537852 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:28:57.537852 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:28:57.537852 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:28:57.537852 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:28:57.537852 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:28:57.537852 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:28:57.537852 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:28:57.537852 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:28:57.537852 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:28:57.537852 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application with Lock( Aug 24 17:28:57.537852 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:28:57.537852 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:28:57.537852 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:28:57.537852 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:28:57.539035 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:28:57.539035 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:28:57.539035 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:28:57.539035 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:28:57.539035 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:28:57.539035 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:28:57.539035 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:28:57.539035 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:28:57.539035 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:28:57.539035 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:28:57.539035 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 17:28:57.539035 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Aug 24 17:28:57.539035 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 533/1065] 10.222.0.45 () {56 vars in 1120 bytes} [Wed Aug 24 17:28:57 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 17:28:57.544100 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26d9db4-f7ec-4594-b63a-8e410323a380 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:57.610745 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b859476d-8e90-41c3-a25d-47b0aa6b2e41 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:57.649344 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d37e1cd-9f8f-4b37-8afd-8051c88be1c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:57.649991 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d37e1cd-9f8f-4b37-8afd-8051c88be1c1 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:57.650298 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d37e1cd-9f8f-4b37-8afd-8051c88be1c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:57.650572 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d37e1cd-9f8f-4b37-8afd-8051c88be1c1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:57.655162 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26d9db4-f7ec-4594-b63a-8e410323a380 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:57.661464 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d37e1cd-9f8f-4b37-8afd-8051c88be1c1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:57.662987 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d37e1cd-9f8f-4b37-8afd-8051c88be1c1 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:57.678420 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application [None req-1d37e1cd-9f8f-4b37-8afd-8051c88be1c1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:28:57.678420 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:28:57.678420 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:28:57.678420 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:28:57.678420 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:28:57.678420 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:28:57.678420 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:28:57.678420 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:28:57.678420 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:28:57.678420 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:28:57.678420 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:28:57.678420 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:28:57.678420 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:28:57.678420 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:28:57.678420 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:28:57.678420 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:28:57.679827 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:28:57.679827 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:28:57.679827 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:28:57.679827 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:28:57.679827 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:28:57.679827 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:28:57.679827 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:28:57.679827 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application with Lock( Aug 24 17:28:57.679827 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:28:57.679827 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:28:57.679827 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:28:57.679827 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:28:57.679827 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:28:57.679827 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:28:57.679827 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:28:57.679827 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:28:57.679827 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:28:57.681259 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:28:57.681259 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:28:57.681259 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Aug 24 17:28:57.681259 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 533/1066] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 17:28:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 17:28:57.689529 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d37e1cd-9f8f-4b37-8afd-8051c88be1c1 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:57.692978 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:57.700198 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0a0d56-fac3-4f13-af0a-754baea4cefa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:57.700786 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0a0d56-fac3-4f13-af0a-754baea4cefa None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:57.701061 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0a0d56-fac3-4f13-af0a-754baea4cefa None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:57.701326 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0a0d56-fac3-4f13-af0a-754baea4cefa None admin] PATH_INFO: `/v3/roles` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:57.702119 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0a0d56-fac3-4f13-af0a-754baea4cefa None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:57.702422 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0a0d56-fac3-4f13-af0a-754baea4cefa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:57.703962 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0a0d56-fac3-4f13-af0a-754baea4cefa None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:57.712588 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 534/1067] 10.222.0.45 () {56 vars in 1115 bytes} [Wed Aug 24 17:28:57 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:28:57.718166 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0a0d56-fac3-4f13-af0a-754baea4cefa None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:57.759144 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:57.817080 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d37e1cd-9f8f-4b37-8afd-8051c88be1c1 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:57.831562 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0a0d56-fac3-4f13-af0a-754baea4cefa None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:57.853129 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17ea97a-2322-4e45-9eb7-36cd6b06c984 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:57.853741 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17ea97a-2322-4e45-9eb7-36cd6b06c984 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:57.854545 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17ea97a-2322-4e45-9eb7-36cd6b06c984 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:57.854822 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17ea97a-2322-4e45-9eb7-36cd6b06c984 None admin] PATH_INFO: `/v3/domains` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:57.855912 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17ea97a-2322-4e45-9eb7-36cd6b06c984 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:57.856419 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17ea97a-2322-4e45-9eb7-36cd6b06c984 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:57.858282 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17ea97a-2322-4e45-9eb7-36cd6b06c984 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:57.868185 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94970acb-ca5d-4095-b31d-f390e397751d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:57.869181 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94970acb-ca5d-4095-b31d-f390e397751d None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:57.870144 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94970acb-ca5d-4095-b31d-f390e397751d None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:57.870580 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94970acb-ca5d-4095-b31d-f390e397751d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:57.874145 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 534/1068] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 24 17:28:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:28:57.889715 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94970acb-ca5d-4095-b31d-f390e397751d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:57.891233 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94970acb-ca5d-4095-b31d-f390e397751d None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:57.907307 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application [None req-94970acb-ca5d-4095-b31d-f390e397751d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:28:57.907307 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:28:57.907307 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:28:57.907307 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:28:57.907307 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:28:57.907307 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:28:57.907307 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:28:57.907307 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:28:57.907307 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:28:57.907307 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:28:57.907307 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:28:57.907307 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:28:57.907307 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:28:57.907307 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:28:57.907307 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:28:57.907307 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:28:57.908395 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:28:57.908395 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:28:57.908395 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:28:57.908395 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:28:57.908395 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:28:57.908395 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:28:57.908395 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:28:57.908395 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application with Lock( Aug 24 17:28:57.908395 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:28:57.908395 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:28:57.908395 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:28:57.908395 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:28:57.908395 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:28:57.908395 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:28:57.908395 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:28:57.908395 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:28:57.908395 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:28:57.909373 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:28:57.909373 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:28:57.909373 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Aug 24 17:28:57.909373 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 535/1069] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 17:28:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 17:28:57.916915 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17ea97a-2322-4e45-9eb7-36cd6b06c984 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:58.025885 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17ea97a-2322-4e45-9eb7-36cd6b06c984 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:58.044848 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:58.063695 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad5776a-e9b1-4e0a-abb2-83e485e50feb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:58.064584 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad5776a-e9b1-4e0a-abb2-83e485e50feb None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:58.065071 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad5776a-e9b1-4e0a-abb2-83e485e50feb None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:58.065524 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad5776a-e9b1-4e0a-abb2-83e485e50feb None admin] PATH_INFO: `/v3/domains` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:58.066342 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad5776a-e9b1-4e0a-abb2-83e485e50feb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:58.066891 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad5776a-e9b1-4e0a-abb2-83e485e50feb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:58.068252 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad5776a-e9b1-4e0a-abb2-83e485e50feb None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:58.080684 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 535/1070] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 24 17:28:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:58.086074 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94970acb-ca5d-4095-b31d-f390e397751d None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:58.144349 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:58.206690 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94970acb-ca5d-4095-b31d-f390e397751d None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:58.241693 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45013c2c-4562-4e4c-b45d-d355b41bf03b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:58.242326 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45013c2c-4562-4e4c-b45d-d355b41bf03b None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:58.242606 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45013c2c-4562-4e4c-b45d-d355b41bf03b None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:58.242879 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45013c2c-4562-4e4c-b45d-d355b41bf03b None admin] PATH_INFO: `/v3/users/nova` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:58.250650 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45013c2c-4562-4e4c-b45d-d355b41bf03b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:58.252088 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45013c2c-4562-4e4c-b45d-d355b41bf03b None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:58.263383 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application [None req-45013c2c-4562-4e4c-b45d-d355b41bf03b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 17:28:58.263383 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:28:58.263383 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:28:58.263383 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:28:58.263383 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:28:58.263383 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:28:58.263383 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:28:58.263383 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:28:58.263383 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:28:58.263383 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:28:58.263383 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:28:58.263383 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:28:58.263383 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 17:28:58.263383 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 17:28:58.263383 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 17:28:58.263383 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 17:28:58.264483 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:28:58.264483 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:28:58.264483 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 17:28:58.264483 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:28:58.264483 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 17:28:58.264483 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:28:58.264483 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:28:58.264483 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:28:58.264483 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:28:58.264483 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:28:58.264483 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:28:58.264483 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application with Lock( Aug 24 17:28:58.264483 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:28:58.264483 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:28:58.264483 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:28:58.264483 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:28:58.264483 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:28:58.264483 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:28:58.265469 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:28:58.265469 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:28:58.265469 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 17:28:58.265469 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 17:28:58.265469 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 17:28:58.265469 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 17:28:58.265469 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 17:28:58.265469 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 17:28:58.265469 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 17:28:58.265469 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Aug 24 17:28:58.265469 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 536/1071] 10.222.0.45 () {56 vars in 1093 bytes} [Wed Aug 24 17:28:58 2022] GET /identity/v3/users/nova => generated 82 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:28:58.270262 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad5776a-e9b1-4e0a-abb2-83e485e50feb None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:58.367827 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad5776a-e9b1-4e0a-abb2-83e485e50feb None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:58.399948 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049c23f4-3721-4c1b-963f-c18917363814 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:58.400562 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049c23f4-3721-4c1b-963f-c18917363814 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:58.401382 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049c23f4-3721-4c1b-963f-c18917363814 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:58.401660 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049c23f4-3721-4c1b-963f-c18917363814 None admin] PATH_INFO: `/v3/users` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:58.402382 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049c23f4-3721-4c1b-963f-c18917363814 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:58.402688 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049c23f4-3721-4c1b-963f-c18917363814 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:58.404109 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049c23f4-3721-4c1b-963f-c18917363814 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:58.420929 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 536/1072] 10.222.0.45 () {56 vars in 1133 bytes} [Wed Aug 24 17:28:58 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:58.427043 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45013c2c-4562-4e4c-b45d-d355b41bf03b None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:58.527784 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45013c2c-4562-4e4c-b45d-d355b41bf03b None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:58.530072 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:58.562334 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457af7f9-52c1-4cde-928e-9dc37f35b087 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:58.562951 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457af7f9-52c1-4cde-928e-9dc37f35b087 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:58.563224 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457af7f9-52c1-4cde-928e-9dc37f35b087 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:58.563490 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457af7f9-52c1-4cde-928e-9dc37f35b087 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:58.573554 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457af7f9-52c1-4cde-928e-9dc37f35b087 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:58.574993 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457af7f9-52c1-4cde-928e-9dc37f35b087 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:58.588240 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application [None req-457af7f9-52c1-4cde-928e-9dc37f35b087 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:28:58.588240 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:28:58.588240 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:28:58.588240 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:28:58.588240 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:28:58.588240 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:28:58.588240 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:28:58.588240 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:28:58.588240 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:28:58.588240 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:28:58.588240 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:28:58.588240 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:28:58.588240 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:28:58.588240 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:28:58.588240 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:28:58.588240 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:28:58.589192 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:28:58.589192 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:28:58.589192 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:28:58.589192 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:28:58.589192 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:28:58.589192 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:28:58.589192 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:28:58.589192 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application with Lock( Aug 24 17:28:58.589192 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:28:58.589192 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:28:58.589192 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:28:58.589192 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:28:58.589192 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:28:58.589192 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:28:58.589192 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:28:58.589192 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:28:58.589192 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:28:58.590084 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:28:58.590084 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:28:58.590084 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Aug 24 17:28:58.590084 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 537/1073] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 17:28:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 17:28:58.595268 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049c23f4-3721-4c1b-963f-c18917363814 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:58.630001 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:58.693015 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049c23f4-3721-4c1b-963f-c18917363814 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:58.725317 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c0a6fb-fad3-4aa0-847f-d9c7ce7664ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:58.725926 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c0a6fb-fad3-4aa0-847f-d9c7ce7664ae None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:58.726205 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c0a6fb-fad3-4aa0-847f-d9c7ce7664ae None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:58.726478 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c0a6fb-fad3-4aa0-847f-d9c7ce7664ae None admin] PATH_INFO: `/v3/domains` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:58.727117 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c0a6fb-fad3-4aa0-847f-d9c7ce7664ae None admin] RBAC: Adding query filter params (name=Default) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:58.727419 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c0a6fb-fad3-4aa0-847f-d9c7ce7664ae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:58.728852 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c0a6fb-fad3-4aa0-847f-d9c7ce7664ae None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:58.740768 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 537/1074] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 24 17:28:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:58.746915 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457af7f9-52c1-4cde-928e-9dc37f35b087 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:58.849212 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457af7f9-52c1-4cde-928e-9dc37f35b087 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:58.882314 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf531b67-2398-439f-a921-627d5094f718 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:58.882871 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf531b67-2398-439f-a921-627d5094f718 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:58.883146 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf531b67-2398-439f-a921-627d5094f718 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:58.883414 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf531b67-2398-439f-a921-627d5094f718 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:58.889775 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf531b67-2398-439f-a921-627d5094f718 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:58.891217 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf531b67-2398-439f-a921-627d5094f718 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:58.902482 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application [None req-cf531b67-2398-439f-a921-627d5094f718 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:28:58.902482 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:28:58.902482 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:28:58.902482 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:28:58.902482 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:28:58.902482 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:28:58.902482 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:28:58.902482 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:28:58.902482 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:28:58.902482 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:28:58.902482 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:28:58.902482 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:28:58.902482 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 17:28:58.902482 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 17:28:58.902482 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 17:28:58.903347 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 17:28:58.903347 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:28:58.903347 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:28:58.903347 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:28:58.903347 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:28:58.903347 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:28:58.903347 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:28:58.903347 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:28:58.903347 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application with Lock( Aug 24 17:28:58.903347 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:28:58.903347 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:28:58.903347 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:28:58.903347 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:28:58.903347 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:28:58.903347 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:28:58.903347 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:28:58.903347 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:28:58.904235 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 17:28:58.904235 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 17:28:58.904235 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 17:28:58.904235 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 17:28:58.904235 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 17:28:58.904235 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 17:28:58.904235 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:28:58.904235 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Aug 24 17:28:58.904235 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 538/1075] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Aug 24 17:28:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 17:28:58.909383 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c0a6fb-fad3-4aa0-847f-d9c7ce7664ae None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:59.000673 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:59.007030 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c0a6fb-fad3-4aa0-847f-d9c7ce7664ae None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:59.039724 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18984aaa-379a-42eb-9fd4-dea6b125f2f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:59.040326 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18984aaa-379a-42eb-9fd4-dea6b125f2f6 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:59.040636 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18984aaa-379a-42eb-9fd4-dea6b125f2f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:59.040909 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18984aaa-379a-42eb-9fd4-dea6b125f2f6 None admin] PATH_INFO: `/v3/projects` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:59.042221 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18984aaa-379a-42eb-9fd4-dea6b125f2f6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:59.042530 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18984aaa-379a-42eb-9fd4-dea6b125f2f6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:59.043957 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18984aaa-379a-42eb-9fd4-dea6b125f2f6 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:59.050156 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a65d09-35f3-46b5-8f02-b5d8d3044203 None None] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:59.050442 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a65d09-35f3-46b5-8f02-b5d8d3044203 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:59.050716 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a65d09-35f3-46b5-8f02-b5d8d3044203 None None] PATH_INFO: `/` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:59.052046 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 539/1076] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:28:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 17:28:59.055823 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 538/1077] 10.222.0.45 () {56 vars in 1148 bytes} [Wed Aug 24 17:28:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:59.059936 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cec9f9-d412-4975-81c8-4969c1d7de4a None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:59.060217 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cec9f9-d412-4975-81c8-4969c1d7de4a None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:59.060583 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cec9f9-d412-4975-81c8-4969c1d7de4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:59.061626 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18984aaa-379a-42eb-9fd4-dea6b125f2f6 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:59.078221 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-f1cec9f9-d412-4975-81c8-4969c1d7de4a None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:28:59.079419 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-f1cec9f9-d412-4975-81c8-4969c1d7de4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:28:59.090449 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:59.148350 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 540/1078] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:28:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:28:59.158956 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18984aaa-379a-42eb-9fd4-dea6b125f2f6 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:59.192825 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09317cd-7673-403c-831a-491df38f7372 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:59.193483 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09317cd-7673-403c-831a-491df38f7372 None admin] REQUEST_METHOD: `PUT` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:59.193765 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09317cd-7673-403c-831a-491df38f7372 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:59.194041 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09317cd-7673-403c-831a-491df38f7372 None admin] PATH_INFO: `/v3/projects/2ad8a32bd9d2434097bd08b111914ca1/users/053b7ce6d6f649ed91a26e434dab805e/roles/d6d7b68e2dee4176bb8712ffbac4808f` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:59.196163 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09317cd-7673-403c-831a-491df38f7372 None admin] RBAC: Authorizing `identity:create_grant(project_id=2ad8a32bd9d2434097bd08b111914ca1, user_id=053b7ce6d6f649ed91a26e434dab805e, role_id=d6d7b68e2dee4176bb8712ffbac4808f)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:59.197648 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09317cd-7673-403c-831a-491df38f7372 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:59.209979 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 539/1079] 10.222.0.45 () {58 vars in 1439 bytes} [Wed Aug 24 17:28:59 2022] PUT /identity/v3/projects/2ad8a32bd9d2434097bd08b111914ca1/users/053b7ce6d6f649ed91a26e434dab805e/roles/d6d7b68e2dee4176bb8712ffbac4808f => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:28:59.420519 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cec9f9-d412-4975-81c8-4969c1d7de4a None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:59.516761 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cec9f9-d412-4975-81c8-4969c1d7de4a None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:59.549999 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03800ba2-3a47-4635-9289-d2974efda126 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:59.550622 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03800ba2-3a47-4635-9289-d2974efda126 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:59.550899 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03800ba2-3a47-4635-9289-d2974efda126 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:59.551173 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03800ba2-3a47-4635-9289-d2974efda126 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:59.560891 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03800ba2-3a47-4635-9289-d2974efda126 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:28:59.565524 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application [None req-03800ba2-3a47-4635-9289-d2974efda126 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:28:59.565524 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:28:59.565524 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:28:59.565524 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:28:59.565524 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:28:59.565524 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:28:59.565524 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:28:59.565524 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:28:59.565524 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:28:59.565524 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:28:59.565524 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:28:59.565524 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:28:59.565524 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:28:59.565524 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:28:59.565524 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:28:59.565524 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:28:59.566690 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:28:59.566690 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:28:59.566690 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:28:59.566690 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:28:59.566690 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:28:59.566690 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:28:59.566690 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:28:59.566690 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:28:59.566690 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:28:59.566690 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:28:59.566690 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:28:59.566690 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application with Lock( Aug 24 17:28:59.566690 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:28:59.566690 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:28:59.566690 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:28:59.566690 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:28:59.567521 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:28:59.567521 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:28:59.567521 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:28:59.567521 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:28:59.567521 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:28:59.567521 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:28:59.567521 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:28:59.567521 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:28:59.567521 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:28:59.567521 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:28:59.567521 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:28:59.567521 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Aug 24 17:28:59.567521 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 541/1080] 10.222.0.45 () {56 vars in 1096 bytes} [Wed Aug 24 17:28:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:28:59.573349 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09317cd-7673-403c-831a-491df38f7372 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:59.591235 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:59.672937 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09317cd-7673-403c-831a-491df38f7372 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:59.706805 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314c2ffd-d016-4651-ac7d-00b92dd12b21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:59.707395 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314c2ffd-d016-4651-ac7d-00b92dd12b21 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:59.707676 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314c2ffd-d016-4651-ac7d-00b92dd12b21 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:59.707948 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314c2ffd-d016-4651-ac7d-00b92dd12b21 None admin] PATH_INFO: `/v3/roles` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:59.708643 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314c2ffd-d016-4651-ac7d-00b92dd12b21 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:59.708950 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314c2ffd-d016-4651-ac7d-00b92dd12b21 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:59.710311 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314c2ffd-d016-4651-ac7d-00b92dd12b21 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:59.718260 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 540/1081] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Aug 24 17:28:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:59.724108 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03800ba2-3a47-4635-9289-d2974efda126 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:59.733545 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:59.825435 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03800ba2-3a47-4635-9289-d2974efda126 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:59.859822 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a141be-d9df-453a-abb6-2007256888d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:59.860699 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a141be-d9df-453a-abb6-2007256888d9 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:59.861037 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a141be-d9df-453a-abb6-2007256888d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:59.861310 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a141be-d9df-453a-abb6-2007256888d9 None admin] PATH_INFO: `/v3/users/16f4741af0d3432db93cb693fcbd7f52` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:59.862820 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a141be-d9df-453a-abb6-2007256888d9 None admin] RBAC: Authorizing `identity:get_user(user_id=16f4741af0d3432db93cb693fcbd7f52)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:59.864794 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a141be-d9df-453a-abb6-2007256888d9 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:59.871715 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 542/1082] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Aug 24 17:28:59 2022] GET /identity/v3/users/16f4741af0d3432db93cb693fcbd7f52 => generated 285 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:59.872910 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314c2ffd-d016-4651-ac7d-00b92dd12b21 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:59.948173 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:59.969400 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314c2ffd-d016-4651-ac7d-00b92dd12b21 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:00.002355 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd97e4cd-406a-45dd-94bb-6770ad4a492a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:00.002974 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd97e4cd-406a-45dd-94bb-6770ad4a492a None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:00.003256 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd97e4cd-406a-45dd-94bb-6770ad4a492a None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:00.003528 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd97e4cd-406a-45dd-94bb-6770ad4a492a None admin] PATH_INFO: `/v3/projects/13b5590b4c0c4eeb84f953f2dcea1bbe` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:00.004669 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd97e4cd-406a-45dd-94bb-6770ad4a492a None admin] RBAC: Authorizing `identity:get_project(project_id=13b5590b4c0c4eeb84f953f2dcea1bbe)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:00.006093 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd97e4cd-406a-45dd-94bb-6770ad4a492a None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:00.007696 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 541/1083] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Aug 24 17:28:59 2022] GET /identity/v3/projects/13b5590b4c0c4eeb84f953f2dcea1bbe => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:00.014321 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a141be-d9df-453a-abb6-2007256888d9 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:00.086321 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:00.111471 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a141be-d9df-453a-abb6-2007256888d9 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:00.145660 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fd20d7-272c-498b-b791-1a2e6bdce222 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:00.146327 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fd20d7-272c-498b-b791-1a2e6bdce222 None admin] REQUEST_METHOD: `PUT` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:00.146607 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fd20d7-272c-498b-b791-1a2e6bdce222 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:00.146882 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fd20d7-272c-498b-b791-1a2e6bdce222 None admin] PATH_INFO: `/v3/projects/13b5590b4c0c4eeb84f953f2dcea1bbe/users/16f4741af0d3432db93cb693fcbd7f52/roles/e411ad9eb96e4e8d85b59b5b700ada7b` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:00.148811 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fd20d7-272c-498b-b791-1a2e6bdce222 None admin] RBAC: Authorizing `identity:create_grant(project_id=13b5590b4c0c4eeb84f953f2dcea1bbe, user_id=16f4741af0d3432db93cb693fcbd7f52, role_id=e411ad9eb96e4e8d85b59b5b700ada7b)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:00.150248 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fd20d7-272c-498b-b791-1a2e6bdce222 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:00.164231 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 543/1084] 10.222.0.45 () {58 vars in 1439 bytes} [Wed Aug 24 17:29:00 2022] PUT /identity/v3/projects/13b5590b4c0c4eeb84f953f2dcea1bbe/users/16f4741af0d3432db93cb693fcbd7f52/roles/e411ad9eb96e4e8d85b59b5b700ada7b => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:29:01.104212 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5122d8a2-07df-4e84-8e8b-180a0e0e47c5 None None] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:01.104774 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5122d8a2-07df-4e84-8e8b-180a0e0e47c5 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:01.104774 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5122d8a2-07df-4e84-8e8b-180a0e0e47c5 None None] PATH_INFO: `/` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:01.106037 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 542/1085] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:29:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:29:01.111701 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f3a78e-8489-4bd9-941d-0a326728d1fd None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:01.112105 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f3a78e-8489-4bd9-941d-0a326728d1fd None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:01.112244 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f3a78e-8489-4bd9-941d-0a326728d1fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:01.129667 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-09f3a78e-8489-4bd9-941d-0a326728d1fd None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:29:01.130835 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-09f3a78e-8489-4bd9-941d-0a326728d1fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:29:01.196344 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 544/1086] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:29:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:29:01.463319 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5122d8a2-07df-4e84-8e8b-180a0e0e47c5 None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:01.563072 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5122d8a2-07df-4e84-8e8b-180a0e0e47c5 None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:01.595542 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f89b8da-5512-4e0e-ab37-1c959f70e8cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:01.596171 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f89b8da-5512-4e0e-ab37-1c959f70e8cf None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:01.596449 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f89b8da-5512-4e0e-ab37-1c959f70e8cf None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:01.596746 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f89b8da-5512-4e0e-ab37-1c959f70e8cf None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:01.607258 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f89b8da-5512-4e0e-ab37-1c959f70e8cf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:29:01.611974 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application [None req-0f89b8da-5512-4e0e-ab37-1c959f70e8cf None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 17:29:01.611974 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:29:01.611974 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:29:01.611974 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:29:01.611974 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:29:01.611974 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:29:01.611974 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:29:01.611974 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:29:01.611974 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:29:01.611974 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:29:01.611974 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:29:01.611974 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:29:01.611974 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:29:01.611974 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:29:01.611974 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:29:01.611974 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:29:01.613072 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:29:01.613072 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:29:01.613072 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:29:01.613072 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:29:01.613072 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:29:01.613072 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:29:01.613072 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:29:01.613072 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:29:01.613072 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:29:01.613072 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:29:01.613072 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:29:01.613072 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application with Lock( Aug 24 17:29:01.613072 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:29:01.613072 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:29:01.613072 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:29:01.613072 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:29:01.613915 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:29:01.613915 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:29:01.613915 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:29:01.613915 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:29:01.613915 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:29:01.613915 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:29:01.613915 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:29:01.613915 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:29:01.613915 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:29:01.613915 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:29:01.613915 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 17:29:01.613915 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Aug 24 17:29:01.613915 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 543/1087] 10.222.0.45 () {56 vars in 1120 bytes} [Wed Aug 24 17:29:01 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:29:01.619036 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f3a78e-8489-4bd9-941d-0a326728d1fd None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:01.678717 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:01.716044 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f3a78e-8489-4bd9-941d-0a326728d1fd None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:01.749935 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8320663a-d41e-44bf-a2a8-fa67a326414d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:01.750540 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8320663a-d41e-44bf-a2a8-fa67a326414d None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:01.750815 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8320663a-d41e-44bf-a2a8-fa67a326414d None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:01.751084 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8320663a-d41e-44bf-a2a8-fa67a326414d None admin] PATH_INFO: `/v3/roles` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:01.751745 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8320663a-d41e-44bf-a2a8-fa67a326414d None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:29:01.752048 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8320663a-d41e-44bf-a2a8-fa67a326414d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:01.753441 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8320663a-d41e-44bf-a2a8-fa67a326414d None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:01.761208 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 545/1088] 10.222.0.45 () {56 vars in 1115 bytes} [Wed Aug 24 17:29:01 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:01.766875 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f89b8da-5512-4e0e-ab37-1c959f70e8cf None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:01.778124 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:01.867130 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f89b8da-5512-4e0e-ab37-1c959f70e8cf None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:01.899828 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf75fc4-216b-4fbd-bf78-2f4a7c09b874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:01.900486 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf75fc4-216b-4fbd-bf78-2f4a7c09b874 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:01.901330 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf75fc4-216b-4fbd-bf78-2f4a7c09b874 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:01.901605 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf75fc4-216b-4fbd-bf78-2f4a7c09b874 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:01.911287 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf75fc4-216b-4fbd-bf78-2f4a7c09b874 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:01.912755 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf75fc4-216b-4fbd-bf78-2f4a7c09b874 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:01.927276 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application [None req-adf75fc4-216b-4fbd-bf78-2f4a7c09b874 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:29:01.927276 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:29:01.927276 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:29:01.927276 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:29:01.927276 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:29:01.927276 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:29:01.927276 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:29:01.927276 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:29:01.927276 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:29:01.927276 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:29:01.927276 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:29:01.927276 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:29:01.927276 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:29:01.927276 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:29:01.927276 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:29:01.927276 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:29:01.933225 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:29:01.933225 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:29:01.933225 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:29:01.933225 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:29:01.933225 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:29:01.933225 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:29:01.933225 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:29:01.933225 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application with Lock( Aug 24 17:29:01.933225 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:29:01.933225 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:29:01.933225 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:29:01.933225 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:29:01.933225 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:29:01.933225 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:29:01.933225 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:29:01.933225 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:29:01.933225 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:29:01.934189 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:29:01.934189 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:29:01.934189 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Aug 24 17:29:01.934189 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 544/1089] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 17:29:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 17:29:01.934432 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8320663a-d41e-44bf-a2a8-fa67a326414d None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:02.033074 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8320663a-d41e-44bf-a2a8-fa67a326414d None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:02.035077 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29284d50-f6ed-41c8-a48e-6ffb39939536 None None] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:02.035367 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29284d50-f6ed-41c8-a48e-6ffb39939536 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:02.035638 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29284d50-f6ed-41c8-a48e-6ffb39939536 None None] PATH_INFO: `/` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:02.037015 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 545/1090] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:29:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 17:29:02.042761 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1232a3a2-b008-4825-accc-aaa3ef7404b8 None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:02.043038 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1232a3a2-b008-4825-accc-aaa3ef7404b8 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:02.043313 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1232a3a2-b008-4825-accc-aaa3ef7404b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:02.059937 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:02.061826 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-1232a3a2-b008-4825-accc-aaa3ef7404b8 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:29:02.063107 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-1232a3a2-b008-4825-accc-aaa3ef7404b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:29:02.068920 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1af9be-5b1c-41e4-a9d8-542dbb0316ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:02.069525 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1af9be-5b1c-41e4-a9d8-542dbb0316ed None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:02.069805 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1af9be-5b1c-41e4-a9d8-542dbb0316ed None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:02.070078 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1af9be-5b1c-41e4-a9d8-542dbb0316ed None admin] PATH_INFO: `/v3/domains` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:02.070727 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1af9be-5b1c-41e4-a9d8-542dbb0316ed None admin] RBAC: Adding query filter params (name=Default) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:29:02.071027 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1af9be-5b1c-41e4-a9d8-542dbb0316ed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:02.072392 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1af9be-5b1c-41e4-a9d8-542dbb0316ed None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:02.084475 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 546/1091] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 24 17:29:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:02.090562 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1af9be-5b1c-41e4-a9d8-542dbb0316ed None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:02.119932 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:02.134070 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 546/1092] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:29:02 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 17:29:02.187664 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1af9be-5b1c-41e4-a9d8-542dbb0316ed None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:02.220442 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7c2154-e4a4-4073-8870-6abcd1a677bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:02.221113 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7c2154-e4a4-4073-8870-6abcd1a677bd None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:02.221391 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7c2154-e4a4-4073-8870-6abcd1a677bd None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:02.221662 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7c2154-e4a4-4073-8870-6abcd1a677bd None admin] PATH_INFO: `/v3/users/nova` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:02.228383 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7c2154-e4a4-4073-8870-6abcd1a677bd None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:02.229866 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7c2154-e4a4-4073-8870-6abcd1a677bd None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:02.241158 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application [None req-ec7c2154-e4a4-4073-8870-6abcd1a677bd None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 17:29:02.241158 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:29:02.241158 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:29:02.241158 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:29:02.241158 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:29:02.241158 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:29:02.241158 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:29:02.241158 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:29:02.241158 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:29:02.241158 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:29:02.241158 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:29:02.241158 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:29:02.241158 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 17:29:02.241158 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 17:29:02.241158 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 17:29:02.241158 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 17:29:02.242277 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:29:02.242277 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:29:02.242277 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 17:29:02.242277 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:29:02.242277 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 17:29:02.242277 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:29:02.242277 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:29:02.242277 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:29:02.242277 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:29:02.242277 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:29:02.242277 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:29:02.242277 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application with Lock( Aug 24 17:29:02.242277 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:29:02.242277 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:29:02.242277 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:29:02.242277 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:29:02.242277 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:29:02.242277 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:29:02.243219 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:29:02.243219 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:29:02.243219 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 17:29:02.243219 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 17:29:02.243219 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 17:29:02.243219 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 17:29:02.243219 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 17:29:02.243219 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 17:29:02.243219 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 17:29:02.243219 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Aug 24 17:29:02.243219 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 547/1093] 10.222.0.45 () {56 vars in 1093 bytes} [Wed Aug 24 17:29:02 2022] GET /identity/v3/users/nova => generated 82 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 17:29:02.248540 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1232a3a2-b008-4825-accc-aaa3ef7404b8 None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:02.342072 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1232a3a2-b008-4825-accc-aaa3ef7404b8 None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:02.374700 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888fdd78-bcca-479b-884c-50da1e6947e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:02.375348 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888fdd78-bcca-479b-884c-50da1e6947e9 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:02.375631 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888fdd78-bcca-479b-884c-50da1e6947e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:02.375902 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888fdd78-bcca-479b-884c-50da1e6947e9 None admin] PATH_INFO: `/v3/users` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:02.376651 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888fdd78-bcca-479b-884c-50da1e6947e9 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:29:02.376957 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888fdd78-bcca-479b-884c-50da1e6947e9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:02.378373 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888fdd78-bcca-479b-884c-50da1e6947e9 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:02.394781 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 547/1094] 10.222.0.45 () {56 vars in 1133 bytes} [Wed Aug 24 17:29:02 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:29:02.400229 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888fdd78-bcca-479b-884c-50da1e6947e9 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:02.400381 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7c2154-e4a4-4073-8870-6abcd1a677bd None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:02.445028 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:02.448287 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:02.498235 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888fdd78-bcca-479b-884c-50da1e6947e9 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:02.501202 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7c2154-e4a4-4073-8870-6abcd1a677bd None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:02.535529 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8356a5-dc1c-4fb6-a26f-a8c0cf33c272 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:02.536139 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8356a5-dc1c-4fb6-a26f-a8c0cf33c272 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:02.536416 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8356a5-dc1c-4fb6-a26f-a8c0cf33c272 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:02.536723 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8356a5-dc1c-4fb6-a26f-a8c0cf33c272 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:02.539602 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0517a8b-c32f-47b1-a4b1-b15bab9f5d17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:02.540239 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0517a8b-c32f-47b1-a4b1-b15bab9f5d17 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:02.540524 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0517a8b-c32f-47b1-a4b1-b15bab9f5d17 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:02.540824 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0517a8b-c32f-47b1-a4b1-b15bab9f5d17 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:02.551748 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0517a8b-c32f-47b1-a4b1-b15bab9f5d17 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:29:02.556386 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application [None req-a0517a8b-c32f-47b1-a4b1-b15bab9f5d17 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:29:02.556386 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:29:02.556386 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:29:02.556386 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:29:02.556386 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:29:02.556386 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:29:02.556386 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:29:02.556386 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:29:02.556386 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:29:02.556386 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:29:02.556386 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:29:02.556386 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:29:02.556386 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:29:02.556386 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:29:02.556386 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:29:02.556386 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:29:02.557409 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:29:02.557409 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:29:02.557409 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:29:02.557409 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:29:02.557409 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:29:02.557409 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:29:02.557409 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:29:02.557409 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:29:02.557409 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:29:02.557409 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:29:02.557409 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:29:02.557409 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application with Lock( Aug 24 17:29:02.557409 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:29:02.557409 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:29:02.557409 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:29:02.557409 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:29:02.558239 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:29:02.558239 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:29:02.558239 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:29:02.558239 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:29:02.558239 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:29:02.558239 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:29:02.558239 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:29:02.558239 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:29:02.558239 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:29:02.558239 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:29:02.558239 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:29:02.558239 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Aug 24 17:29:02.558239 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 548/1095] 10.222.0.45 () {56 vars in 1096 bytes} [Wed Aug 24 17:29:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:29:02.563609 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0517a8b-c32f-47b1-a4b1-b15bab9f5d17 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:02.603153 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8356a5-dc1c-4fb6-a26f-a8c0cf33c272 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:02.605185 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8356a5-dc1c-4fb6-a26f-a8c0cf33c272 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:02.647569 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:02.660650 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0517a8b-c32f-47b1-a4b1-b15bab9f5d17 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:02.667896 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:02.698384 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03ef6d9-27bc-4a35-a495-6c0071b8f076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:02.698990 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03ef6d9-27bc-4a35-a495-6c0071b8f076 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:02.699270 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03ef6d9-27bc-4a35-a495-6c0071b8f076 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:02.699542 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03ef6d9-27bc-4a35-a495-6c0071b8f076 None admin] PATH_INFO: `/v3/roles` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:02.700203 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03ef6d9-27bc-4a35-a495-6c0071b8f076 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:29:02.700505 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03ef6d9-27bc-4a35-a495-6c0071b8f076 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:02.700905 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 548/1096] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Aug 24 17:29:02 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 24 17:29:02.701941 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03ef6d9-27bc-4a35-a495-6c0071b8f076 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:02.707072 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8356a5-dc1c-4fb6-a26f-a8c0cf33c272 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:02.715649 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 549/1097] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Aug 24 17:29:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:29:02.716657 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03ef6d9-27bc-4a35-a495-6c0071b8f076 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:02.805960 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8356a5-dc1c-4fb6-a26f-a8c0cf33c272 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:02.814191 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03ef6d9-27bc-4a35-a495-6c0071b8f076 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:02.840147 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a0b0ae-a856-44fb-9f47-093f3db17101 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:02.841116 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a0b0ae-a856-44fb-9f47-093f3db17101 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:02.841400 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a0b0ae-a856-44fb-9f47-093f3db17101 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:02.841671 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a0b0ae-a856-44fb-9f47-093f3db17101 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:02.847571 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f48168-dbc8-4ee9-a78a-f9084040e65b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:02.848189 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f48168-dbc8-4ee9-a78a-f9084040e65b None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:02.848465 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f48168-dbc8-4ee9-a78a-f9084040e65b None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:02.848759 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f48168-dbc8-4ee9-a78a-f9084040e65b None admin] PATH_INFO: `/v3/users/16f4741af0d3432db93cb693fcbd7f52` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:02.849870 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f48168-dbc8-4ee9-a78a-f9084040e65b None admin] RBAC: Authorizing `identity:get_user(user_id=16f4741af0d3432db93cb693fcbd7f52)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:02.851287 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f48168-dbc8-4ee9-a78a-f9084040e65b None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:02.851394 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a0b0ae-a856-44fb-9f47-093f3db17101 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:02.852795 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a0b0ae-a856-44fb-9f47-093f3db17101 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:02.852887 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 550/1098] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Aug 24 17:29:02 2022] GET /identity/v3/users/16f4741af0d3432db93cb693fcbd7f52 => generated 285 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:02.858976 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f48168-dbc8-4ee9-a78a-f9084040e65b None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:02.860519 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:02.868955 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application [None req-16a0b0ae-a856-44fb-9f47-093f3db17101 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:29:02.868955 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:29:02.868955 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:29:02.868955 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:29:02.868955 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:29:02.868955 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:29:02.868955 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:29:02.868955 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:29:02.868955 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:29:02.868955 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:29:02.868955 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:29:02.868955 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:29:02.868955 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:29:02.868955 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:29:02.868955 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:29:02.868955 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:29:02.870130 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:29:02.870130 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:29:02.870130 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:29:02.870130 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:29:02.870130 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:29:02.870130 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:29:02.870130 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:29:02.870130 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application with Lock( Aug 24 17:29:02.870130 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:29:02.870130 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:29:02.870130 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:29:02.870130 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:29:02.870130 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:29:02.870130 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:29:02.870130 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:29:02.870130 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:29:02.870130 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:29:02.871198 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:29:02.871198 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:29:02.871198 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Aug 24 17:29:02.871198 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 549/1099] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 17:29:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 17:29:02.875862 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a0b0ae-a856-44fb-9f47-093f3db17101 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:02.895062 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:02.958236 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f48168-dbc8-4ee9-a78a-f9084040e65b None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:02.974326 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a0b0ae-a856-44fb-9f47-093f3db17101 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:02.992247 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e120307-f003-4b1b-8b53-d915eb8077ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:02.992885 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e120307-f003-4b1b-8b53-d915eb8077ea None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:02.993166 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e120307-f003-4b1b-8b53-d915eb8077ea None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:02.993441 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e120307-f003-4b1b-8b53-d915eb8077ea None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:03.007004 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf8961e-6d6a-4109-8739-a9aacd6a01f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:03.007603 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf8961e-6d6a-4109-8739-a9aacd6a01f5 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:03.007882 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf8961e-6d6a-4109-8739-a9aacd6a01f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:03.008163 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf8961e-6d6a-4109-8739-a9aacd6a01f5 None admin] PATH_INFO: `/v3/domains` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:03.008829 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf8961e-6d6a-4109-8739-a9aacd6a01f5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:29:03.009135 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf8961e-6d6a-4109-8739-a9aacd6a01f5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:03.011017 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf8961e-6d6a-4109-8739-a9aacd6a01f5 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:03.022593 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 550/1100] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 24 17:29:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:29:03.028824 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf8961e-6d6a-4109-8739-a9aacd6a01f5 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:03.059355 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e120307-f003-4b1b-8b53-d915eb8077ea None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:03.060823 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e120307-f003-4b1b-8b53-d915eb8077ea None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:03.085462 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:03.093888 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:03.127542 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf8961e-6d6a-4109-8739-a9aacd6a01f5 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:03.161446 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc5aa29-a833-4d6a-a8c0-6ebdbe3743af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:03.161647 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 551/1101] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Aug 24 17:29:02 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 24 17:29:03.162075 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc5aa29-a833-4d6a-a8c0-6ebdbe3743af None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:03.162357 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc5aa29-a833-4d6a-a8c0-6ebdbe3743af None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:03.162711 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc5aa29-a833-4d6a-a8c0-6ebdbe3743af None admin] PATH_INFO: `/v3/projects/service` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:03.167299 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e120307-f003-4b1b-8b53-d915eb8077ea None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:03.169554 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc5aa29-a833-4d6a-a8c0-6ebdbe3743af None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:03.171013 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc5aa29-a833-4d6a-a8c0-6ebdbe3743af None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:03.181972 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application [None req-dbc5aa29-a833-4d6a-a8c0-6ebdbe3743af None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:29:03.181972 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:29:03.181972 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:29:03.181972 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:29:03.181972 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:29:03.181972 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:29:03.181972 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:29:03.181972 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:29:03.181972 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:29:03.181972 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:29:03.181972 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:29:03.181972 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:29:03.181972 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 17:29:03.181972 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 17:29:03.181972 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 17:29:03.183422 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 17:29:03.183422 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:29:03.183422 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:29:03.183422 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:29:03.183422 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:29:03.183422 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:29:03.183422 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:29:03.183422 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:29:03.183422 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application with Lock( Aug 24 17:29:03.183422 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:29:03.183422 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:29:03.183422 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:29:03.183422 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:29:03.183422 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:29:03.183422 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:29:03.183422 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:29:03.183422 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:29:03.184315 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 17:29:03.184315 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 17:29:03.184315 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 17:29:03.184315 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 17:29:03.184315 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 17:29:03.184315 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 17:29:03.184315 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:29:03.184315 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Aug 24 17:29:03.184315 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 551/1102] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Aug 24 17:29:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 17:29:03.189197 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc5aa29-a833-4d6a-a8c0-6ebdbe3743af None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:03.262808 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e120307-f003-4b1b-8b53-d915eb8077ea None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:03.285441 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc5aa29-a833-4d6a-a8c0-6ebdbe3743af None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:03.294586 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a25e39-442f-4a7b-b1a2-5ccfd4fc0035 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:03.295203 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a25e39-442f-4a7b-b1a2-5ccfd4fc0035 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:03.295479 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a25e39-442f-4a7b-b1a2-5ccfd4fc0035 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:03.295745 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a25e39-442f-4a7b-b1a2-5ccfd4fc0035 None admin] PATH_INFO: `/v3/projects/13b5590b4c0c4eeb84f953f2dcea1bbe` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:03.296876 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a25e39-442f-4a7b-b1a2-5ccfd4fc0035 None admin] RBAC: Authorizing `identity:get_project(project_id=13b5590b4c0c4eeb84f953f2dcea1bbe)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:03.298289 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a25e39-442f-4a7b-b1a2-5ccfd4fc0035 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:03.299885 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 552/1103] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Aug 24 17:29:03 2022] GET /identity/v3/projects/13b5590b4c0c4eeb84f953f2dcea1bbe => generated 311 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:29:03.305599 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:03.305892 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a25e39-442f-4a7b-b1a2-5ccfd4fc0035 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:03.309955 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:03.324404 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f31558-d1ff-429c-b0d4-7efa2a7519fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:03.325055 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f31558-d1ff-429c-b0d4-7efa2a7519fc None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:03.325337 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f31558-d1ff-429c-b0d4-7efa2a7519fc None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:03.325615 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f31558-d1ff-429c-b0d4-7efa2a7519fc None admin] PATH_INFO: `/v3/projects` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:03.326338 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f31558-d1ff-429c-b0d4-7efa2a7519fc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:29:03.326647 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f31558-d1ff-429c-b0d4-7efa2a7519fc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:03.328075 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f31558-d1ff-429c-b0d4-7efa2a7519fc None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:03.339889 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 552/1104] 10.222.0.45 () {56 vars in 1148 bytes} [Wed Aug 24 17:29:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:29:03.345770 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f31558-d1ff-429c-b0d4-7efa2a7519fc None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:03.404811 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a25e39-442f-4a7b-b1a2-5ccfd4fc0035 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:03.437841 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3ef9ff-e85e-45de-a4d6-593148e6e4b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:03.438437 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3ef9ff-e85e-45de-a4d6-593148e6e4b9 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:03.438710 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3ef9ff-e85e-45de-a4d6-593148e6e4b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:03.438977 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3ef9ff-e85e-45de-a4d6-593148e6e4b9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:03.439691 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3ef9ff-e85e-45de-a4d6-593148e6e4b9 None admin] RBAC: Adding query filter params (role.id=e411ad9eb96e4e8d85b59b5b700ada7b, scope.project.id=13b5590b4c0c4eeb84f953f2dcea1bbe, user.id=16f4741af0d3432db93cb693fcbd7f52) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:29:03.439991 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3ef9ff-e85e-45de-a4d6-593148e6e4b9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:03.441436 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3ef9ff-e85e-45de-a4d6-593148e6e4b9 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:03.443005 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f31558-d1ff-429c-b0d4-7efa2a7519fc None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:03.446957 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 553/1105] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Aug 24 17:29:03 2022] GET /identity/v3/role_assignments?user.id=16f4741af0d3432db93cb693fcbd7f52&scope.project.id=13b5590b4c0c4eeb84f953f2dcea1bbe&role.id=e411ad9eb96e4e8d85b59b5b700ada7b => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:29:03.477827 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b50db0-647f-476e-a568-f4cfac237e22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:03.478446 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b50db0-647f-476e-a568-f4cfac237e22 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:03.478724 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b50db0-647f-476e-a568-f4cfac237e22 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:03.479542 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b50db0-647f-476e-a568-f4cfac237e22 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:03.480282 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b50db0-647f-476e-a568-f4cfac237e22 None admin] RBAC: Adding query filter params (role.id=d6d7b68e2dee4176bb8712ffbac4808f, scope.project.id=2ad8a32bd9d2434097bd08b111914ca1, user.id=053b7ce6d6f649ed91a26e434dab805e) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:29:03.480616 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b50db0-647f-476e-a568-f4cfac237e22 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:03.482339 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b50db0-647f-476e-a568-f4cfac237e22 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:03.488946 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 553/1106] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Aug 24 17:29:03 2022] GET /identity/v3/role_assignments?user.id=053b7ce6d6f649ed91a26e434dab805e&scope.project.id=2ad8a32bd9d2434097bd08b111914ca1&role.id=d6d7b68e2dee4176bb8712ffbac4808f => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:29:05.387360 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd041d6-a60b-496a-8825-d4e3eff6fdbe None None] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:05.387920 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd041d6-a60b-496a-8825-d4e3eff6fdbe None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:05.387920 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd041d6-a60b-496a-8825-d4e3eff6fdbe None None] PATH_INFO: `/` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:05.389367 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 554/1107] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:29:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:29:05.394508 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858adcee-0fb0-43c8-ab43-dc570ff74d7c None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:05.394923 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858adcee-0fb0-43c8-ab43-dc570ff74d7c None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:05.395055 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858adcee-0fb0-43c8-ab43-dc570ff74d7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:05.414171 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-858adcee-0fb0-43c8-ab43-dc570ff74d7c None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:29:05.415343 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-858adcee-0fb0-43c8-ab43-dc570ff74d7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:29:05.448842 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:05.479979 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 554/1108] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:29:05 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:29:05.749681 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd041d6-a60b-496a-8825-d4e3eff6fdbe None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:05.816109 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:05.849547 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd041d6-a60b-496a-8825-d4e3eff6fdbe None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:05.881641 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90ec1f2-96ef-4387-9331-5e5852d28af6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:05.882260 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90ec1f2-96ef-4387-9331-5e5852d28af6 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:05.882536 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90ec1f2-96ef-4387-9331-5e5852d28af6 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:05.882806 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90ec1f2-96ef-4387-9331-5e5852d28af6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:05.883902 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90ec1f2-96ef-4387-9331-5e5852d28af6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:05.885888 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90ec1f2-96ef-4387-9331-5e5852d28af6 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:05.887422 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 555/1109] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 17:29:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:05.893151 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858adcee-0fb0-43c8-ab43-dc570ff74d7c None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:05.986125 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858adcee-0fb0-43c8-ab43-dc570ff74d7c None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:06.018361 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7834cc-497c-48c1-a685-893631106c86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:06.018889 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7834cc-497c-48c1-a685-893631106c86 None admin] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:06.019163 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7834cc-497c-48c1-a685-893631106c86 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:06.019430 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7834cc-497c-48c1-a685-893631106c86 None admin] PATH_INFO: `/v3/users` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:06.020072 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7834cc-497c-48c1-a685-893631106c86 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:06.021501 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7834cc-497c-48c1-a685-893631106c86 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:06.042251 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 555/1110] 10.222.0.45 () {60 vars in 1132 bytes} [Wed Aug 24 17:29:05 2022] POST /identity/v3/users => generated 286 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:29:07.948834 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324f5626-3c46-4193-85c5-0a64976d4e2c None None] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:07.949346 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324f5626-3c46-4193-85c5-0a64976d4e2c None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:07.949346 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324f5626-3c46-4193-85c5-0a64976d4e2c None None] PATH_INFO: `/` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:07.950684 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 556/1111] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:29:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:29:07.955928 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffbf149-4fa5-46f3-82e9-e1c8217cf82a None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:07.956333 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffbf149-4fa5-46f3-82e9-e1c8217cf82a None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:07.956481 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffbf149-4fa5-46f3-82e9-e1c8217cf82a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:07.974034 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-dffbf149-4fa5-46f3-82e9-e1c8217cf82a None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:29:07.975212 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-dffbf149-4fa5-46f3-82e9-e1c8217cf82a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:29:07.998214 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:08.040715 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 556/1112] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:29:07 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 17:29:08.313567 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324f5626-3c46-4193-85c5-0a64976d4e2c None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:08.411529 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324f5626-3c46-4193-85c5-0a64976d4e2c None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:08.443458 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3ee651-429a-412a-aeb5-b4a7280932ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:08.444068 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3ee651-429a-412a-aeb5-b4a7280932ed None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:08.444343 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3ee651-429a-412a-aeb5-b4a7280932ed None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:08.444636 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3ee651-429a-412a-aeb5-b4a7280932ed None admin] PATH_INFO: `/v3/roles/148f135e4a134697a032be04bc9faa4c` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:08.445996 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3ee651-429a-412a-aeb5-b4a7280932ed None admin] RBAC: Authorizing `identity:get_role(role_id=148f135e4a134697a032be04bc9faa4c)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:08.447348 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3ee651-429a-412a-aeb5-b4a7280932ed None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:08.448686 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 557/1113] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Aug 24 17:29:08 2022] GET /identity/v3/roles/148f135e4a134697a032be04bc9faa4c => generated 223 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:08.454232 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffbf149-4fa5-46f3-82e9-e1c8217cf82a None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:08.550048 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffbf149-4fa5-46f3-82e9-e1c8217cf82a None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:08.582426 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0121267-4986-46df-a1d6-4f183ef07822 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:08.582973 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0121267-4986-46df-a1d6-4f183ef07822 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:08.583248 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0121267-4986-46df-a1d6-4f183ef07822 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:08.583517 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0121267-4986-46df-a1d6-4f183ef07822 None admin] PATH_INFO: `/v3/users/7e6e6bd384b343908541b1ccdfa39560` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:08.596361 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0121267-4986-46df-a1d6-4f183ef07822 None admin] RBAC: Authorizing `identity:get_user(user_id=7e6e6bd384b343908541b1ccdfa39560)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:08.597805 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0121267-4986-46df-a1d6-4f183ef07822 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:08.599368 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 557/1114] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Aug 24 17:29:08 2022] GET /identity/v3/users/7e6e6bd384b343908541b1ccdfa39560 => generated 286 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:08.605879 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3ee651-429a-412a-aeb5-b4a7280932ed None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:08.609880 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:08.701956 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3ee651-429a-412a-aeb5-b4a7280932ed None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:08.734648 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b6b2a8-b14c-4454-8782-f59747f864c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:08.735242 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b6b2a8-b14c-4454-8782-f59747f864c4 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:08.735521 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b6b2a8-b14c-4454-8782-f59747f864c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:08.735795 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b6b2a8-b14c-4454-8782-f59747f864c4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:08.799763 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b6b2a8-b14c-4454-8782-f59747f864c4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:08.801243 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b6b2a8-b14c-4454-8782-f59747f864c4 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:08.803772 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:08.896342 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 558/1115] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Aug 24 17:29:08 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 17:29:08.902309 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0121267-4986-46df-a1d6-4f183ef07822 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:08.919965 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:08.997959 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0121267-4986-46df-a1d6-4f183ef07822 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:09.030768 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe83f34-58d9-4ff8-8c9e-0bea3525b26d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:09.031382 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe83f34-58d9-4ff8-8c9e-0bea3525b26d None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:09.031659 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe83f34-58d9-4ff8-8c9e-0bea3525b26d None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:09.031932 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe83f34-58d9-4ff8-8c9e-0bea3525b26d None admin] PATH_INFO: `/v3/projects/13b5590b4c0c4eeb84f953f2dcea1bbe` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:09.033054 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe83f34-58d9-4ff8-8c9e-0bea3525b26d None admin] RBAC: Authorizing `identity:get_project(project_id=13b5590b4c0c4eeb84f953f2dcea1bbe)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:09.034472 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe83f34-58d9-4ff8-8c9e-0bea3525b26d None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:09.036072 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 558/1116] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Aug 24 17:29:08 2022] GET /identity/v3/projects/13b5590b4c0c4eeb84f953f2dcea1bbe => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:09.042081 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b6b2a8-b14c-4454-8782-f59747f864c4 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:09.136828 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b6b2a8-b14c-4454-8782-f59747f864c4 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:09.139051 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:09.170388 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c009199-a470-4826-b51d-4be4fea5cb2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:09.171056 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c009199-a470-4826-b51d-4be4fea5cb2e None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:09.171538 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c009199-a470-4826-b51d-4be4fea5cb2e None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:09.171538 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c009199-a470-4826-b51d-4be4fea5cb2e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:09.172173 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c009199-a470-4826-b51d-4be4fea5cb2e None admin] RBAC: Adding query filter params (role.id=148f135e4a134697a032be04bc9faa4c, scope.project.id=13b5590b4c0c4eeb84f953f2dcea1bbe, user.id=7e6e6bd384b343908541b1ccdfa39560) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:29:09.172478 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c009199-a470-4826-b51d-4be4fea5cb2e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:09.173943 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c009199-a470-4826-b51d-4be4fea5cb2e None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:09.179283 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 559/1117] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Aug 24 17:29:09 2022] GET /identity/v3/role_assignments?user.id=7e6e6bd384b343908541b1ccdfa39560&scope.project.id=13b5590b4c0c4eeb84f953f2dcea1bbe&role.id=148f135e4a134697a032be04bc9faa4c => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:11.011665 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494692df-0040-4bb6-9caa-759e7e159fc0 None None] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:11.012175 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494692df-0040-4bb6-9caa-759e7e159fc0 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:11.012260 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494692df-0040-4bb6-9caa-759e7e159fc0 None None] PATH_INFO: `/` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:11.013746 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 559/1118] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:29:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:29:11.018734 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1468ed-f589-4ba1-9334-de44db333985 None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:11.019135 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1468ed-f589-4ba1-9334-de44db333985 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:11.019282 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1468ed-f589-4ba1-9334-de44db333985 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:11.036752 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-3a1468ed-f589-4ba1-9334-de44db333985 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:29:11.037932 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-3a1468ed-f589-4ba1-9334-de44db333985 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:29:11.100999 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 560/1119] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:29:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:29:11.371876 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494692df-0040-4bb6-9caa-759e7e159fc0 None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:11.453100 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:11.469525 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494692df-0040-4bb6-9caa-759e7e159fc0 None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:11.502497 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a05836-0d94-4a81-b143-637b1c2aaea8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:11.503101 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a05836-0d94-4a81-b143-637b1c2aaea8 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:11.503378 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a05836-0d94-4a81-b143-637b1c2aaea8 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:11.503643 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a05836-0d94-4a81-b143-637b1c2aaea8 None admin] PATH_INFO: `/v3/roles/148f135e4a134697a032be04bc9faa4c` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:11.505030 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a05836-0d94-4a81-b143-637b1c2aaea8 None admin] RBAC: Authorizing `identity:get_role(role_id=148f135e4a134697a032be04bc9faa4c)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:11.506373 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a05836-0d94-4a81-b143-637b1c2aaea8 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:11.507652 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 560/1120] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Aug 24 17:29:11 2022] GET /identity/v3/roles/148f135e4a134697a032be04bc9faa4c => generated 223 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:11.513387 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1468ed-f589-4ba1-9334-de44db333985 None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:11.605103 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:11.609299 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1468ed-f589-4ba1-9334-de44db333985 None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:11.641468 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c331fbd0-a12d-4c53-8c41-f2a5cb456881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:11.642089 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c331fbd0-a12d-4c53-8c41-f2a5cb456881 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:11.642367 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c331fbd0-a12d-4c53-8c41-f2a5cb456881 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:11.642640 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c331fbd0-a12d-4c53-8c41-f2a5cb456881 None admin] PATH_INFO: `/v3/users/7e6e6bd384b343908541b1ccdfa39560` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:11.643737 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c331fbd0-a12d-4c53-8c41-f2a5cb456881 None admin] RBAC: Authorizing `identity:get_user(user_id=7e6e6bd384b343908541b1ccdfa39560)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:11.645191 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c331fbd0-a12d-4c53-8c41-f2a5cb456881 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:11.646725 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 561/1121] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Aug 24 17:29:11 2022] GET /identity/v3/users/7e6e6bd384b343908541b1ccdfa39560 => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:11.652813 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a05836-0d94-4a81-b143-637b1c2aaea8 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:11.747604 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a05836-0d94-4a81-b143-637b1c2aaea8 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:11.780606 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b347ba-7324-4d33-b8f1-446fce7d70c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:11.781227 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b347ba-7324-4d33-b8f1-446fce7d70c4 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:11.781505 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b347ba-7324-4d33-b8f1-446fce7d70c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:11.781774 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b347ba-7324-4d33-b8f1-446fce7d70c4 None admin] PATH_INFO: `/v3/projects/13b5590b4c0c4eeb84f953f2dcea1bbe` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:11.782871 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b347ba-7324-4d33-b8f1-446fce7d70c4 None admin] RBAC: Authorizing `identity:get_project(project_id=13b5590b4c0c4eeb84f953f2dcea1bbe)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:11.784288 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b347ba-7324-4d33-b8f1-446fce7d70c4 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:11.785917 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 561/1122] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Aug 24 17:29:11 2022] GET /identity/v3/projects/13b5590b4c0c4eeb84f953f2dcea1bbe => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:11.791666 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c331fbd0-a12d-4c53-8c41-f2a5cb456881 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:11.887781 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c331fbd0-a12d-4c53-8c41-f2a5cb456881 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:11.920361 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f8456e-08ca-402a-8607-36d92d490117 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:11.920976 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f8456e-08ca-402a-8607-36d92d490117 None admin] REQUEST_METHOD: `PUT` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:11.921253 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f8456e-08ca-402a-8607-36d92d490117 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:11.921525 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f8456e-08ca-402a-8607-36d92d490117 None admin] PATH_INFO: `/v3/projects/13b5590b4c0c4eeb84f953f2dcea1bbe/users/7e6e6bd384b343908541b1ccdfa39560/roles/148f135e4a134697a032be04bc9faa4c` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:11.923637 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f8456e-08ca-402a-8607-36d92d490117 None admin] RBAC: Authorizing `identity:create_grant(project_id=13b5590b4c0c4eeb84f953f2dcea1bbe, user_id=7e6e6bd384b343908541b1ccdfa39560, role_id=148f135e4a134697a032be04bc9faa4c)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:11.925092 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f8456e-08ca-402a-8607-36d92d490117 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:11.937132 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 562/1123] 10.222.0.45 () {58 vars in 1439 bytes} [Wed Aug 24 17:29:11 2022] PUT /identity/v3/projects/13b5590b4c0c4eeb84f953f2dcea1bbe/users/7e6e6bd384b343908541b1ccdfa39560/roles/148f135e4a134697a032be04bc9faa4c => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:29:13.792494 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bb2572-c5fd-4a14-95c0-f6b17edc3653 None None] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:13.793055 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bb2572-c5fd-4a14-95c0-f6b17edc3653 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:13.793055 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bb2572-c5fd-4a14-95c0-f6b17edc3653 None None] PATH_INFO: `/` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:13.794415 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 562/1124] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:29:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:29:13.799750 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf85dcbc-cf55-428e-bf20-cb5ba0d02f2a None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:13.800122 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf85dcbc-cf55-428e-bf20-cb5ba0d02f2a None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:13.800298 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf85dcbc-cf55-428e-bf20-cb5ba0d02f2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:13.817741 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-bf85dcbc-cf55-428e-bf20-cb5ba0d02f2a None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:29:13.818905 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-bf85dcbc-cf55-428e-bf20-cb5ba0d02f2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:29:13.838910 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:13.884837 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 563/1125] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:29:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:29:14.155063 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bb2572-c5fd-4a14-95c0-f6b17edc3653 None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:14.176019 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:14.252026 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bb2572-c5fd-4a14-95c0-f6b17edc3653 None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:14.284669 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32b29f9-9dff-4636-92c0-47dc20ac4d66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:14.285278 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32b29f9-9dff-4636-92c0-47dc20ac4d66 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:14.285551 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32b29f9-9dff-4636-92c0-47dc20ac4d66 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:14.285820 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32b29f9-9dff-4636-92c0-47dc20ac4d66 None admin] PATH_INFO: `/v3/roles/148f135e4a134697a032be04bc9faa4c` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:14.287173 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32b29f9-9dff-4636-92c0-47dc20ac4d66 None admin] RBAC: Authorizing `identity:get_role(role_id=148f135e4a134697a032be04bc9faa4c)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:14.288521 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32b29f9-9dff-4636-92c0-47dc20ac4d66 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:14.289827 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 563/1126] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Aug 24 17:29:14 2022] GET /identity/v3/roles/148f135e4a134697a032be04bc9faa4c => generated 223 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:14.295550 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf85dcbc-cf55-428e-bf20-cb5ba0d02f2a None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:14.391568 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf85dcbc-cf55-428e-bf20-cb5ba0d02f2a None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:14.423613 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920db835-5634-420a-964b-7fd763a2c137 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:14.424176 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920db835-5634-420a-964b-7fd763a2c137 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:14.424454 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920db835-5634-420a-964b-7fd763a2c137 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:14.424761 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920db835-5634-420a-964b-7fd763a2c137 None admin] PATH_INFO: `/v3/users/7e6e6bd384b343908541b1ccdfa39560` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:14.425872 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920db835-5634-420a-964b-7fd763a2c137 None admin] RBAC: Authorizing `identity:get_user(user_id=7e6e6bd384b343908541b1ccdfa39560)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:14.427300 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920db835-5634-420a-964b-7fd763a2c137 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:14.428865 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 564/1127] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Aug 24 17:29:14 2022] GET /identity/v3/users/7e6e6bd384b343908541b1ccdfa39560 => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:14.434788 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32b29f9-9dff-4636-92c0-47dc20ac4d66 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:14.518114 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:14.529943 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32b29f9-9dff-4636-92c0-47dc20ac4d66 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:14.561919 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b2c546-2343-4e2d-a2bc-8ec3ebc8ea74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:14.562501 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b2c546-2343-4e2d-a2bc-8ec3ebc8ea74 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:14.562775 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b2c546-2343-4e2d-a2bc-8ec3ebc8ea74 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:14.563042 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b2c546-2343-4e2d-a2bc-8ec3ebc8ea74 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:14.627609 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b2c546-2343-4e2d-a2bc-8ec3ebc8ea74 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:14.629069 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b2c546-2343-4e2d-a2bc-8ec3ebc8ea74 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:14.710706 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:14.722787 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 564/1128] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Aug 24 17:29:14 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 17:29:14.728722 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920db835-5634-420a-964b-7fd763a2c137 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:14.753982 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:14.828515 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920db835-5634-420a-964b-7fd763a2c137 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:14.861380 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15274ac-22ae-4ffa-b2dd-f651b3f36a12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:14.861999 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15274ac-22ae-4ffa-b2dd-f651b3f36a12 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:14.862279 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15274ac-22ae-4ffa-b2dd-f651b3f36a12 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:14.862553 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15274ac-22ae-4ffa-b2dd-f651b3f36a12 None admin] PATH_INFO: `/v3/projects/13b5590b4c0c4eeb84f953f2dcea1bbe` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:14.863661 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15274ac-22ae-4ffa-b2dd-f651b3f36a12 None admin] RBAC: Authorizing `identity:get_project(project_id=13b5590b4c0c4eeb84f953f2dcea1bbe)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:14.865124 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15274ac-22ae-4ffa-b2dd-f651b3f36a12 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:14.866738 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 565/1129] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Aug 24 17:29:14 2022] GET /identity/v3/projects/13b5590b4c0c4eeb84f953f2dcea1bbe => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:14.873085 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b2c546-2343-4e2d-a2bc-8ec3ebc8ea74 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:14.966417 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b2c546-2343-4e2d-a2bc-8ec3ebc8ea74 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:14.998005 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e2d2dd-7e84-4902-84cd-eed73aaa2315 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:14.998588 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e2d2dd-7e84-4902-84cd-eed73aaa2315 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:14.998866 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e2d2dd-7e84-4902-84cd-eed73aaa2315 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:14.999135 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e2d2dd-7e84-4902-84cd-eed73aaa2315 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:14.999850 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e2d2dd-7e84-4902-84cd-eed73aaa2315 None admin] RBAC: Adding query filter params (role.id=148f135e4a134697a032be04bc9faa4c, scope.project.id=13b5590b4c0c4eeb84f953f2dcea1bbe, user.id=7e6e6bd384b343908541b1ccdfa39560) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:29:15.000155 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e2d2dd-7e84-4902-84cd-eed73aaa2315 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:15.001590 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e2d2dd-7e84-4902-84cd-eed73aaa2315 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:15.006997 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 565/1130] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Aug 24 17:29:14 2022] GET /identity/v3/role_assignments?user.id=7e6e6bd384b343908541b1ccdfa39560&scope.project.id=13b5590b4c0c4eeb84f953f2dcea1bbe&role.id=148f135e4a134697a032be04bc9faa4c => generated 608 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:16.935569 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3a8752-96ab-422b-bfce-1ce978cd2a43 None None] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:16.936067 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3a8752-96ab-422b-bfce-1ce978cd2a43 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:16.936183 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3a8752-96ab-422b-bfce-1ce978cd2a43 None None] PATH_INFO: `/` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:16.937481 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 566/1131] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:29:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:29:16.942745 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22938492-a43e-4ff9-8343-b1a826b3a866 None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:16.943163 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22938492-a43e-4ff9-8343-b1a826b3a866 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:16.943298 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22938492-a43e-4ff9-8343-b1a826b3a866 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:16.963396 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-22938492-a43e-4ff9-8343-b1a826b3a866 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:29:16.965508 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-22938492-a43e-4ff9-8343-b1a826b3a866 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:29:17.010680 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:17.030284 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 566/1132] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:29:16 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:29:17.294940 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3a8752-96ab-422b-bfce-1ce978cd2a43 None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:17.383520 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:17.393259 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3a8752-96ab-422b-bfce-1ce978cd2a43 None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:17.426070 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a31f928-dbfe-4615-a32f-6dc063f5742a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:17.426685 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a31f928-dbfe-4615-a32f-6dc063f5742a None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:17.426963 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a31f928-dbfe-4615-a32f-6dc063f5742a None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:17.427235 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a31f928-dbfe-4615-a32f-6dc063f5742a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:17.428331 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a31f928-dbfe-4615-a32f-6dc063f5742a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:17.429794 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a31f928-dbfe-4615-a32f-6dc063f5742a None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:17.431325 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 567/1133] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 17:29:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:17.437445 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22938492-a43e-4ff9-8343-b1a826b3a866 None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:17.530314 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22938492-a43e-4ff9-8343-b1a826b3a866 None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:17.562360 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfe8ff6-b993-4b4f-9673-31de72d8307e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:17.562890 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfe8ff6-b993-4b4f-9673-31de72d8307e None admin] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:17.563166 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfe8ff6-b993-4b4f-9673-31de72d8307e None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:17.563431 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfe8ff6-b993-4b4f-9673-31de72d8307e None admin] PATH_INFO: `/v3/projects` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:17.564067 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfe8ff6-b993-4b4f-9673-31de72d8307e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:17.565494 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfe8ff6-b993-4b4f-9673-31de72d8307e None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:17.591413 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 567/1134] 10.222.0.45 () {60 vars in 1141 bytes} [Wed Aug 24 17:29:17 2022] POST /identity/v3/projects => generated 311 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:29:19.475860 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581e5731-0e0f-422d-8549-91b542d46547 None None] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:19.477835 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581e5731-0e0f-422d-8549-91b542d46547 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:19.477835 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581e5731-0e0f-422d-8549-91b542d46547 None None] PATH_INFO: `/` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:19.478042 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 568/1135] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:29:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:29:19.484068 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b26d110-af51-49d4-8a4e-243cc9a3cabd None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:19.484510 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b26d110-af51-49d4-8a4e-243cc9a3cabd None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:19.484655 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b26d110-af51-49d4-8a4e-243cc9a3cabd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:19.502732 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-7b26d110-af51-49d4-8a4e-243cc9a3cabd None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:29:19.503943 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-7b26d110-af51-49d4-8a4e-243cc9a3cabd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:29:19.529171 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:19.569088 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 568/1136] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:29:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:29:19.836569 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581e5731-0e0f-422d-8549-91b542d46547 None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:19.934704 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581e5731-0e0f-422d-8549-91b542d46547 None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:19.967878 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578f9380-5a08-400e-9be2-c006b6b8fa0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:19.968528 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578f9380-5a08-400e-9be2-c006b6b8fa0c None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:19.968837 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578f9380-5a08-400e-9be2-c006b6b8fa0c None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:19.969111 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578f9380-5a08-400e-9be2-c006b6b8fa0c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:19.970217 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578f9380-5a08-400e-9be2-c006b6b8fa0c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:19.971654 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578f9380-5a08-400e-9be2-c006b6b8fa0c None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:19.973223 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 569/1137] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 17:29:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:19.978784 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b26d110-af51-49d4-8a4e-243cc9a3cabd None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:20.072528 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b26d110-af51-49d4-8a4e-243cc9a3cabd None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:20.105077 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182c47e6-7dfb-4a6b-a7bf-c2ec1a651c41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:20.105670 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182c47e6-7dfb-4a6b-a7bf-c2ec1a651c41 None admin] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:20.105949 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182c47e6-7dfb-4a6b-a7bf-c2ec1a651c41 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:20.106217 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182c47e6-7dfb-4a6b-a7bf-c2ec1a651c41 None admin] PATH_INFO: `/v3/users` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:20.106854 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182c47e6-7dfb-4a6b-a7bf-c2ec1a651c41 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:20.108529 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182c47e6-7dfb-4a6b-a7bf-c2ec1a651c41 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:20.128708 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 569/1138] 10.222.0.45 () {60 vars in 1132 bytes} [Wed Aug 24 17:29:19 2022] POST /identity/v3/users => generated 286 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:29:22.041025 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a96ab1-057b-48c9-9f71-6b03f948fdec None None] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:22.041576 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a96ab1-057b-48c9-9f71-6b03f948fdec None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:22.041576 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a96ab1-057b-48c9-9f71-6b03f948fdec None None] PATH_INFO: `/` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:22.042881 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 570/1139] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:29:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:29:22.048139 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490fc88d-8e1f-4648-a7b6-6ef10f45377c None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:22.048592 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490fc88d-8e1f-4648-a7b6-6ef10f45377c None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:22.048733 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490fc88d-8e1f-4648-a7b6-6ef10f45377c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:22.066826 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-490fc88d-8e1f-4648-a7b6-6ef10f45377c None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:29:22.067947 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-490fc88d-8e1f-4648-a7b6-6ef10f45377c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:29:22.081936 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:22.133527 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 570/1140] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:29:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:29:22.402033 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a96ab1-057b-48c9-9f71-6b03f948fdec None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:22.429347 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:22.500683 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a96ab1-057b-48c9-9f71-6b03f948fdec None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:22.533706 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ff1e7e-cd4b-427c-8d9e-01bd2c5c9b94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:22.534320 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ff1e7e-cd4b-427c-8d9e-01bd2c5c9b94 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:22.534599 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ff1e7e-cd4b-427c-8d9e-01bd2c5c9b94 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:22.534875 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ff1e7e-cd4b-427c-8d9e-01bd2c5c9b94 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:22.544843 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ff1e7e-cd4b-427c-8d9e-01bd2c5c9b94 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:29:22.549448 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application [None req-73ff1e7e-cd4b-427c-8d9e-01bd2c5c9b94 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:29:22.549448 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:29:22.549448 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:29:22.549448 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:29:22.549448 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:29:22.549448 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:29:22.549448 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:29:22.549448 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:29:22.549448 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:29:22.549448 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:29:22.549448 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:29:22.549448 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:29:22.549448 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:29:22.549448 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:29:22.549448 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:29:22.549448 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:29:22.552742 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:29:22.552742 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:29:22.552742 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:29:22.552742 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:29:22.552742 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:29:22.552742 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:29:22.552742 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:29:22.552742 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:29:22.552742 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:29:22.552742 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:29:22.552742 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:29:22.552742 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application with Lock( Aug 24 17:29:22.552742 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:29:22.552742 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:29:22.552742 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:29:22.552742 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:29:22.553845 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:29:22.553845 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:29:22.553845 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:29:22.553845 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:29:22.553845 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:29:22.553845 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:29:22.553845 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:29:22.553845 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:29:22.553845 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:29:22.553845 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:29:22.553845 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:29:22.553845 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Aug 24 17:29:22.553845 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 571/1141] 10.222.0.45 () {56 vars in 1096 bytes} [Wed Aug 24 17:29:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:29:22.556386 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490fc88d-8e1f-4648-a7b6-6ef10f45377c None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:22.657962 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490fc88d-8e1f-4648-a7b6-6ef10f45377c None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:22.690324 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c117d0c-4bf8-4792-b389-c4259930e8fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:22.690924 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c117d0c-4bf8-4792-b389-c4259930e8fe None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:22.691204 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c117d0c-4bf8-4792-b389-c4259930e8fe None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:22.691477 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c117d0c-4bf8-4792-b389-c4259930e8fe None admin] PATH_INFO: `/v3/roles` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:22.692133 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c117d0c-4bf8-4792-b389-c4259930e8fe None admin] RBAC: Adding query filter params (name=admin) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:29:22.692435 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c117d0c-4bf8-4792-b389-c4259930e8fe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:22.693822 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c117d0c-4bf8-4792-b389-c4259930e8fe None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:22.702077 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 571/1142] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Aug 24 17:29:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:22.707794 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ff1e7e-cd4b-427c-8d9e-01bd2c5c9b94 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:22.792698 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:22.805343 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ff1e7e-cd4b-427c-8d9e-01bd2c5c9b94 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:22.840021 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66cc3df-4b31-43e0-a60f-d9fc7f9f95eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:22.840671 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66cc3df-4b31-43e0-a60f-d9fc7f9f95eb None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:22.840956 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66cc3df-4b31-43e0-a60f-d9fc7f9f95eb None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:22.841232 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66cc3df-4b31-43e0-a60f-d9fc7f9f95eb None admin] PATH_INFO: `/v3/users/dde8bbcf1fb34e12ac25dcea354c08ab` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:22.853627 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66cc3df-4b31-43e0-a60f-d9fc7f9f95eb None admin] RBAC: Authorizing `identity:get_user(user_id=dde8bbcf1fb34e12ac25dcea354c08ab)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:22.855069 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66cc3df-4b31-43e0-a60f-d9fc7f9f95eb None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:22.856827 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 572/1143] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Aug 24 17:29:22 2022] GET /identity/v3/users/dde8bbcf1fb34e12ac25dcea354c08ab => generated 286 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:22.862859 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c117d0c-4bf8-4792-b389-c4259930e8fe None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:22.878794 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:22.961088 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c117d0c-4bf8-4792-b389-c4259930e8fe None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:22.993864 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e600998-e39c-4d5c-a6b8-491f0d0600c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:22.994452 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e600998-e39c-4d5c-a6b8-491f0d0600c7 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:22.995269 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e600998-e39c-4d5c-a6b8-491f0d0600c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:22.995545 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e600998-e39c-4d5c-a6b8-491f0d0600c7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:23.058304 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e600998-e39c-4d5c-a6b8-491f0d0600c7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:23.059729 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e600998-e39c-4d5c-a6b8-491f0d0600c7 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:23.072765 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:23.152920 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 572/1144] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Aug 24 17:29:22 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 17:29:23.158762 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66cc3df-4b31-43e0-a60f-d9fc7f9f95eb None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:23.253946 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66cc3df-4b31-43e0-a60f-d9fc7f9f95eb None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:23.285982 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1dba3df-5c8a-4cc8-a118-bb2a7f20425c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:23.286601 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dba3df-5c8a-4cc8-a118-bb2a7f20425c None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:23.286879 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dba3df-5c8a-4cc8-a118-bb2a7f20425c None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:23.287152 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dba3df-5c8a-4cc8-a118-bb2a7f20425c None admin] PATH_INFO: `/v3/projects/dd58ed611d944d87acf01f8688ccd919` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:23.288255 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1dba3df-5c8a-4cc8-a118-bb2a7f20425c None admin] RBAC: Authorizing `identity:get_project(project_id=dd58ed611d944d87acf01f8688ccd919)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:23.289717 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1dba3df-5c8a-4cc8-a118-bb2a7f20425c None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:23.291309 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 573/1145] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Aug 24 17:29:23 2022] GET /identity/v3/projects/dd58ed611d944d87acf01f8688ccd919 => generated 311 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:23.297786 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e600998-e39c-4d5c-a6b8-491f0d0600c7 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:23.391158 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e600998-e39c-4d5c-a6b8-491f0d0600c7 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:23.403926 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:23.424485 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2882e199-d1dd-498a-b31e-243f9aae7a52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:23.425093 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2882e199-d1dd-498a-b31e-243f9aae7a52 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:23.425369 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2882e199-d1dd-498a-b31e-243f9aae7a52 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:23.425639 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2882e199-d1dd-498a-b31e-243f9aae7a52 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:23.426348 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2882e199-d1dd-498a-b31e-243f9aae7a52 None admin] RBAC: Adding query filter params (role.id=e411ad9eb96e4e8d85b59b5b700ada7b, scope.project.id=dd58ed611d944d87acf01f8688ccd919, user.id=dde8bbcf1fb34e12ac25dcea354c08ab) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:29:23.426650 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2882e199-d1dd-498a-b31e-243f9aae7a52 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:23.428054 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2882e199-d1dd-498a-b31e-243f9aae7a52 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:23.433180 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 573/1146] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Aug 24 17:29:23 2022] GET /identity/v3/role_assignments?user.id=dde8bbcf1fb34e12ac25dcea354c08ab&scope.project.id=dd58ed611d944d87acf01f8688ccd919&role.id=e411ad9eb96e4e8d85b59b5b700ada7b => generated 259 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:25.263002 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786adcf2-74c2-42ba-b7f0-88b4a1947ad7 None None] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:25.263520 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786adcf2-74c2-42ba-b7f0-88b4a1947ad7 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:25.263520 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786adcf2-74c2-42ba-b7f0-88b4a1947ad7 None None] PATH_INFO: `/` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:25.264922 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 574/1147] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:29:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:29:25.269868 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3deff1-8811-4114-9374-a4a0defedf33 None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:25.270275 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3deff1-8811-4114-9374-a4a0defedf33 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:25.270409 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3deff1-8811-4114-9374-a4a0defedf33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:25.287747 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-fe3deff1-8811-4114-9374-a4a0defedf33 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:29:25.288940 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-fe3deff1-8811-4114-9374-a4a0defedf33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:29:25.351756 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 574/1148] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:29:25 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:29:25.612644 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786adcf2-74c2-42ba-b7f0-88b4a1947ad7 None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:25.627956 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:25.710196 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786adcf2-74c2-42ba-b7f0-88b4a1947ad7 None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:25.743578 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ceaf81-98fe-4a07-8a66-e9f5507c4a3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:25.744193 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ceaf81-98fe-4a07-8a66-e9f5507c4a3f None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:25.744472 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ceaf81-98fe-4a07-8a66-e9f5507c4a3f None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:25.744810 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ceaf81-98fe-4a07-8a66-e9f5507c4a3f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:25.754509 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ceaf81-98fe-4a07-8a66-e9f5507c4a3f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:29:25.759155 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application [None req-e8ceaf81-98fe-4a07-8a66-e9f5507c4a3f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:29:25.759155 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:29:25.759155 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:29:25.759155 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:29:25.759155 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:29:25.759155 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:29:25.759155 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:29:25.759155 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:29:25.759155 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:29:25.759155 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:29:25.759155 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:29:25.759155 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:29:25.759155 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:29:25.759155 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:29:25.759155 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:29:25.759155 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:29:25.760235 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:29:25.760235 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:29:25.760235 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:29:25.760235 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:29:25.760235 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:29:25.760235 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:29:25.760235 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:29:25.760235 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:29:25.760235 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:29:25.760235 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:29:25.760235 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:29:25.760235 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application with Lock( Aug 24 17:29:25.760235 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:29:25.760235 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:29:25.760235 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:29:25.760235 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:29:25.765746 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:29:25.765746 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:29:25.765746 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:29:25.765746 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:29:25.765746 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:29:25.765746 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:29:25.765746 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:29:25.765746 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:29:25.765746 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:29:25.765746 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:29:25.765746 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:29:25.765746 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Aug 24 17:29:25.765746 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 575/1149] 10.222.0.45 () {56 vars in 1096 bytes} [Wed Aug 24 17:29:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:29:25.766606 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3deff1-8811-4114-9374-a4a0defedf33 None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:25.862180 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3deff1-8811-4114-9374-a4a0defedf33 None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:25.869142 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:25.894294 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598afaa7-7c72-4817-ac79-8bed67fb20fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:25.894896 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598afaa7-7c72-4817-ac79-8bed67fb20fe None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:25.895172 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598afaa7-7c72-4817-ac79-8bed67fb20fe None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:25.895441 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598afaa7-7c72-4817-ac79-8bed67fb20fe None admin] PATH_INFO: `/v3/roles` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:25.896089 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598afaa7-7c72-4817-ac79-8bed67fb20fe None admin] RBAC: Adding query filter params (name=admin) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:29:25.896387 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598afaa7-7c72-4817-ac79-8bed67fb20fe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:25.897769 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598afaa7-7c72-4817-ac79-8bed67fb20fe None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:25.905868 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 575/1150] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Aug 24 17:29:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:25.911622 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ceaf81-98fe-4a07-8a66-e9f5507c4a3f None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:25.984446 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:26.008012 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ceaf81-98fe-4a07-8a66-e9f5507c4a3f None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:26.040575 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a13aee-b26c-4ae8-8860-c5ae6753279d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:26.041186 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a13aee-b26c-4ae8-8860-c5ae6753279d None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:26.041461 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a13aee-b26c-4ae8-8860-c5ae6753279d None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:26.041729 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a13aee-b26c-4ae8-8860-c5ae6753279d None admin] PATH_INFO: `/v3/users/dde8bbcf1fb34e12ac25dcea354c08ab` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:26.042816 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a13aee-b26c-4ae8-8860-c5ae6753279d None admin] RBAC: Authorizing `identity:get_user(user_id=dde8bbcf1fb34e12ac25dcea354c08ab)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:26.044241 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a13aee-b26c-4ae8-8860-c5ae6753279d None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:26.045797 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 576/1151] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Aug 24 17:29:25 2022] GET /identity/v3/users/dde8bbcf1fb34e12ac25dcea354c08ab => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:26.051315 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598afaa7-7c72-4817-ac79-8bed67fb20fe None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:26.146300 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598afaa7-7c72-4817-ac79-8bed67fb20fe None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:26.178013 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985ee35f-60f2-4773-a2cf-9dca3ef09790 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:26.178617 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985ee35f-60f2-4773-a2cf-9dca3ef09790 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:26.178893 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985ee35f-60f2-4773-a2cf-9dca3ef09790 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:26.179160 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985ee35f-60f2-4773-a2cf-9dca3ef09790 None admin] PATH_INFO: `/v3/projects/dd58ed611d944d87acf01f8688ccd919` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:26.180241 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985ee35f-60f2-4773-a2cf-9dca3ef09790 None admin] RBAC: Authorizing `identity:get_project(project_id=dd58ed611d944d87acf01f8688ccd919)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:26.181672 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985ee35f-60f2-4773-a2cf-9dca3ef09790 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:26.183304 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 576/1152] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Aug 24 17:29:26 2022] GET /identity/v3/projects/dd58ed611d944d87acf01f8688ccd919 => generated 311 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:26.189099 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a13aee-b26c-4ae8-8860-c5ae6753279d None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:26.284500 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a13aee-b26c-4ae8-8860-c5ae6753279d None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:26.316538 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3081cb-43d5-425e-a7e9-ed4bd40574af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:26.317196 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3081cb-43d5-425e-a7e9-ed4bd40574af None admin] REQUEST_METHOD: `PUT` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:26.317471 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3081cb-43d5-425e-a7e9-ed4bd40574af None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:26.317741 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3081cb-43d5-425e-a7e9-ed4bd40574af None admin] PATH_INFO: `/v3/projects/dd58ed611d944d87acf01f8688ccd919/users/dde8bbcf1fb34e12ac25dcea354c08ab/roles/e411ad9eb96e4e8d85b59b5b700ada7b` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:26.319592 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3081cb-43d5-425e-a7e9-ed4bd40574af None admin] RBAC: Authorizing `identity:create_grant(project_id=dd58ed611d944d87acf01f8688ccd919, user_id=dde8bbcf1fb34e12ac25dcea354c08ab, role_id=e411ad9eb96e4e8d85b59b5b700ada7b)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:26.321037 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3081cb-43d5-425e-a7e9-ed4bd40574af None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:26.333933 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 577/1153] 10.222.0.45 () {58 vars in 1439 bytes} [Wed Aug 24 17:29:26 2022] PUT /identity/v3/projects/dd58ed611d944d87acf01f8688ccd919/users/dde8bbcf1fb34e12ac25dcea354c08ab/roles/e411ad9eb96e4e8d85b59b5b700ada7b => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:29:28.183668 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21c02a4-aa52-4a2e-a467-8c1fbaa05669 None None] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:28.184217 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21c02a4-aa52-4a2e-a467-8c1fbaa05669 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:28.184217 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21c02a4-aa52-4a2e-a467-8c1fbaa05669 None None] PATH_INFO: `/` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:28.185586 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 577/1154] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:29:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:29:28.190707 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c735bf-5981-4058-8ae6-15029d38f415 None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:28.191077 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c735bf-5981-4058-8ae6-15029d38f415 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:28.191263 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c735bf-5981-4058-8ae6-15029d38f415 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:28.197561 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:28.211089 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-96c735bf-5981-4058-8ae6-15029d38f415 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:29:28.212287 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-96c735bf-5981-4058-8ae6-15029d38f415 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:29:28.278376 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 578/1155] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:29:28 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:29:28.541551 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21c02a4-aa52-4a2e-a467-8c1fbaa05669 None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:28.577534 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:28.640421 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21c02a4-aa52-4a2e-a467-8c1fbaa05669 None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:28.672035 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe65d31-0436-4e03-8541-22382310d218 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:28.672670 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe65d31-0436-4e03-8541-22382310d218 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:28.672952 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe65d31-0436-4e03-8541-22382310d218 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:28.673223 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe65d31-0436-4e03-8541-22382310d218 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:28.683409 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe65d31-0436-4e03-8541-22382310d218 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:29:28.688052 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application [None req-7fe65d31-0436-4e03-8541-22382310d218 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:29:28.688052 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:29:28.688052 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:29:28.688052 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:29:28.688052 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:29:28.688052 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:29:28.688052 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:29:28.688052 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:29:28.688052 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:29:28.688052 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:29:28.688052 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:29:28.688052 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:29:28.688052 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:29:28.688052 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:29:28.688052 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:29:28.688052 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:29:28.689179 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:29:28.689179 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:29:28.689179 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:29:28.689179 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:29:28.689179 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:29:28.689179 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:29:28.689179 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:29:28.689179 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:29:28.689179 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:29:28.689179 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:29:28.689179 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:29:28.689179 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application with Lock( Aug 24 17:29:28.689179 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:29:28.689179 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:29:28.689179 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:29:28.689179 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:29:28.690040 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:29:28.690040 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:29:28.690040 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:29:28.690040 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:29:28.690040 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:29:28.690040 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:29:28.690040 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:29:28.690040 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:29:28.690040 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:29:28.690040 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:29:28.690040 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:29:28.690040 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Aug 24 17:29:28.690040 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 578/1156] 10.222.0.45 () {56 vars in 1096 bytes} [Wed Aug 24 17:29:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:29:28.696307 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c735bf-5981-4058-8ae6-15029d38f415 None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:28.792082 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c735bf-5981-4058-8ae6-15029d38f415 None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:28.820673 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:28.826606 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee57913-658e-42b6-be1c-b607ccef7aaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:28.827214 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee57913-658e-42b6-be1c-b607ccef7aaf None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:28.827492 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee57913-658e-42b6-be1c-b607ccef7aaf None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:28.827763 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee57913-658e-42b6-be1c-b607ccef7aaf None admin] PATH_INFO: `/v3/roles` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:28.828430 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee57913-658e-42b6-be1c-b607ccef7aaf None admin] RBAC: Adding query filter params (name=admin) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:29:28.828767 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee57913-658e-42b6-be1c-b607ccef7aaf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:28.830136 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee57913-658e-42b6-be1c-b607ccef7aaf None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:28.838077 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 579/1157] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Aug 24 17:29:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:29:28.843836 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe65d31-0436-4e03-8541-22382310d218 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:28.933829 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:28.939955 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe65d31-0436-4e03-8541-22382310d218 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:28.972478 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4ba6f4-fd9c-4936-835d-61607a845470 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:28.973123 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4ba6f4-fd9c-4936-835d-61607a845470 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:28.973401 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4ba6f4-fd9c-4936-835d-61607a845470 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:28.973670 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4ba6f4-fd9c-4936-835d-61607a845470 None admin] PATH_INFO: `/v3/users/dde8bbcf1fb34e12ac25dcea354c08ab` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:28.974773 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4ba6f4-fd9c-4936-835d-61607a845470 None admin] RBAC: Authorizing `identity:get_user(user_id=dde8bbcf1fb34e12ac25dcea354c08ab)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:28.976188 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4ba6f4-fd9c-4936-835d-61607a845470 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:28.977781 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 579/1158] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Aug 24 17:29:28 2022] GET /identity/v3/users/dde8bbcf1fb34e12ac25dcea354c08ab => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:28.984302 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee57913-658e-42b6-be1c-b607ccef7aaf None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:29.080264 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee57913-658e-42b6-be1c-b607ccef7aaf None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:29.112817 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b997f68-e70f-4fd7-881c-fae0e7da6790 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:29.113405 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b997f68-e70f-4fd7-881c-fae0e7da6790 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:29.113684 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b997f68-e70f-4fd7-881c-fae0e7da6790 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:29.113956 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b997f68-e70f-4fd7-881c-fae0e7da6790 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:29.172076 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:29.179542 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b997f68-e70f-4fd7-881c-fae0e7da6790 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:29.181018 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b997f68-e70f-4fd7-881c-fae0e7da6790 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:29.274322 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 580/1159] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Aug 24 17:29:28 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 17:29:29.280165 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4ba6f4-fd9c-4936-835d-61607a845470 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:29.376178 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4ba6f4-fd9c-4936-835d-61607a845470 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:29.408640 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f61afe-48d5-4043-9300-9a5e28f4dd4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:29.409252 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f61afe-48d5-4043-9300-9a5e28f4dd4a None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:29.409530 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f61afe-48d5-4043-9300-9a5e28f4dd4a None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:29.409803 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f61afe-48d5-4043-9300-9a5e28f4dd4a None admin] PATH_INFO: `/v3/projects/dd58ed611d944d87acf01f8688ccd919` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:29.410900 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f61afe-48d5-4043-9300-9a5e28f4dd4a None admin] RBAC: Authorizing `identity:get_project(project_id=dd58ed611d944d87acf01f8688ccd919)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:29.412325 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f61afe-48d5-4043-9300-9a5e28f4dd4a None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:29.413940 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 580/1160] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Aug 24 17:29:29 2022] GET /identity/v3/projects/dd58ed611d944d87acf01f8688ccd919 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:29.420429 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b997f68-e70f-4fd7-881c-fae0e7da6790 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:29.508789 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:29.515634 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b997f68-e70f-4fd7-881c-fae0e7da6790 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:29.548647 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cb9479-449f-4d24-8811-2d82570ec35f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:29.549237 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cb9479-449f-4d24-8811-2d82570ec35f None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:29.549515 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cb9479-449f-4d24-8811-2d82570ec35f None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:29.549788 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cb9479-449f-4d24-8811-2d82570ec35f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:29.550502 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cb9479-449f-4d24-8811-2d82570ec35f None admin] RBAC: Adding query filter params (role.id=e411ad9eb96e4e8d85b59b5b700ada7b, scope.project.id=dd58ed611d944d87acf01f8688ccd919, user.id=dde8bbcf1fb34e12ac25dcea354c08ab) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:29:29.550808 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cb9479-449f-4d24-8811-2d82570ec35f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:29.552220 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cb9479-449f-4d24-8811-2d82570ec35f None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:29.557772 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 581/1161] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Aug 24 17:29:29 2022] GET /identity/v3/role_assignments?user.id=dde8bbcf1fb34e12ac25dcea354c08ab&scope.project.id=dd58ed611d944d87acf01f8688ccd919&role.id=e411ad9eb96e4e8d85b59b5b700ada7b => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:31.453343 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a370991c-549b-404f-895c-080a20af7d82 None None] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:31.453904 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a370991c-549b-404f-895c-080a20af7d82 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:31.453904 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a370991c-549b-404f-895c-080a20af7d82 None None] PATH_INFO: `/` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:31.455217 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 581/1162] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:29:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:29:31.460420 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99194b2-5da2-4dca-bb9b-579e4dc4b3fb None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:31.460867 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99194b2-5da2-4dca-bb9b-579e4dc4b3fb None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:31.460994 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99194b2-5da2-4dca-bb9b-579e4dc4b3fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:31.478451 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-d99194b2-5da2-4dca-bb9b-579e4dc4b3fb None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:29:31.480143 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-d99194b2-5da2-4dca-bb9b-579e4dc4b3fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:29:31.544808 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 582/1163] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:29:31 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:29:31.808738 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a370991c-549b-404f-895c-080a20af7d82 None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:31.839002 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:31.906978 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a370991c-549b-404f-895c-080a20af7d82 None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:31.939475 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dc854c-30e7-482b-b93d-427218138ec7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:31.940068 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dc854c-30e7-482b-b93d-427218138ec7 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:31.940346 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dc854c-30e7-482b-b93d-427218138ec7 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:31.940643 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dc854c-30e7-482b-b93d-427218138ec7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:32.004139 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5dc854c-30e7-482b-b93d-427218138ec7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:32.005608 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5dc854c-30e7-482b-b93d-427218138ec7 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:32.034869 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:32.100409 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 582/1164] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Aug 24 17:29:31 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 17:29:32.106290 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99194b2-5da2-4dca-bb9b-579e4dc4b3fb None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:32.190567 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:32.202995 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99194b2-5da2-4dca-bb9b-579e4dc4b3fb None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:32.235867 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2447fd7f-2e21-49d6-8f85-b3016312b3d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:32.236488 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2447fd7f-2e21-49d6-8f85-b3016312b3d3 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:32.236795 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2447fd7f-2e21-49d6-8f85-b3016312b3d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:32.237070 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2447fd7f-2e21-49d6-8f85-b3016312b3d3 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:32.243442 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2447fd7f-2e21-49d6-8f85-b3016312b3d3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:32.244922 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2447fd7f-2e21-49d6-8f85-b3016312b3d3 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:32.256172 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application [None req-2447fd7f-2e21-49d6-8f85-b3016312b3d3 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 24 17:29:32.256172 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:29:32.256172 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 24 17:29:32.256172 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 24 17:29:32.256172 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 17:29:32.256172 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 17:29:32.256172 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 17:29:32.256172 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 17:29:32.256172 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Aug 24 17:29:32.256172 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Aug 24 17:29:32.256172 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 17:29:32.256172 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Aug 24 17:29:32.256172 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:29:32.256172 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:29:32.256172 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:29:32.256172 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:29:32.256172 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:29:32.257467 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:29:32.257467 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:29:32.257467 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:29:32.257467 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:29:32.257467 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:29:32.257467 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:29:32.257467 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:29:32.257467 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:29:32.257467 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:29:32.257467 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:29:32.257467 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:29:32.257467 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:29:32.257467 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:29:32.257467 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:29:32.257467 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:29:32.257467 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:29:32.258690 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:29:32.258690 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application with Lock( Aug 24 17:29:32.258690 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:29:32.258690 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:29:32.258690 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:29:32.258690 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:29:32.258690 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:29:32.258690 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:29:32.258690 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:29:32.258690 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:29:32.258690 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 24 17:29:32.258690 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:29:32.258690 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 24 17:29:32.258690 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Aug 24 17:29:32.258690 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 583/1165] 10.222.0.45 () {56 vars in 1117 bytes} [Wed Aug 24 17:29:32 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:29:32.263631 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dc854c-30e7-482b-b93d-427218138ec7 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:32.358440 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dc854c-30e7-482b-b93d-427218138ec7 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:32.386396 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:32.392045 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71d0dd0-c578-469c-a336-d17e6e4673d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:32.392737 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71d0dd0-c578-469c-a336-d17e6e4673d2 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:32.393023 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71d0dd0-c578-469c-a336-d17e6e4673d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:32.393298 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71d0dd0-c578-469c-a336-d17e6e4673d2 None admin] PATH_INFO: `/v3/domains` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:32.393943 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71d0dd0-c578-469c-a336-d17e6e4673d2 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:29:32.394250 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71d0dd0-c578-469c-a336-d17e6e4673d2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:32.395605 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71d0dd0-c578-469c-a336-d17e6e4673d2 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:32.402536 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 583/1166] 10.222.0.45 () {56 vars in 1115 bytes} [Wed Aug 24 17:29:32 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:34.245066 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379f4e65-b990-4a57-84e3-d1c5bdbe4eec None None] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:34.245618 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379f4e65-b990-4a57-84e3-d1c5bdbe4eec None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:34.245618 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379f4e65-b990-4a57-84e3-d1c5bdbe4eec None None] PATH_INFO: `/` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:34.247599 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 584/1167] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:29:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:29:34.252732 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d83945-102c-4702-b583-c3a384857dbf None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:34.253162 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d83945-102c-4702-b583-c3a384857dbf None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:34.253287 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d83945-102c-4702-b583-c3a384857dbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:34.270678 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-e8d83945-102c-4702-b583-c3a384857dbf None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:29:34.271854 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-e8d83945-102c-4702-b583-c3a384857dbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:29:34.335071 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 584/1168] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:29:34 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:29:34.602570 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379f4e65-b990-4a57-84e3-d1c5bdbe4eec None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:34.701033 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379f4e65-b990-4a57-84e3-d1c5bdbe4eec None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:34.734269 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733d2b33-818a-4e16-bb97-a8420eaa852f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:34.734861 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733d2b33-818a-4e16-bb97-a8420eaa852f None admin] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:34.735140 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733d2b33-818a-4e16-bb97-a8420eaa852f None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:34.735411 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733d2b33-818a-4e16-bb97-a8420eaa852f None admin] PATH_INFO: `/v3/domains` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:34.736063 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733d2b33-818a-4e16-bb97-a8420eaa852f None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:34.737449 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733d2b33-818a-4e16-bb97-a8420eaa852f None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:34.753423 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 585/1169] 10.222.0.45 () {60 vars in 1138 bytes} [Wed Aug 24 17:29:34 2022] POST /identity/v3/domains => generated 267 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:29:36.691338 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567fbf71-c52e-4310-9dcd-c302c3b28038 None None] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:36.691916 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567fbf71-c52e-4310-9dcd-c302c3b28038 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:36.692163 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567fbf71-c52e-4310-9dcd-c302c3b28038 None None] PATH_INFO: `/` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:36.693998 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 585/1170] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:29:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:29:36.699236 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97029cd1-2106-44f5-b541-6f6584167703 None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:36.699611 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97029cd1-2106-44f5-b541-6f6584167703 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:36.699783 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97029cd1-2106-44f5-b541-6f6584167703 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:36.717182 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-97029cd1-2106-44f5-b541-6f6584167703 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:29:36.718348 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-97029cd1-2106-44f5-b541-6f6584167703 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:29:36.720239 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:36.785486 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 586/1171] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:29:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:29:37.051401 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567fbf71-c52e-4310-9dcd-c302c3b28038 None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:37.147599 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567fbf71-c52e-4310-9dcd-c302c3b28038 None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:37.171952 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:37.180646 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0850aa-05c3-41eb-a0a6-1f77798be0e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:37.181256 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0850aa-05c3-41eb-a0a6-1f77798be0e4 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:37.181532 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0850aa-05c3-41eb-a0a6-1f77798be0e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:37.181801 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0850aa-05c3-41eb-a0a6-1f77798be0e4 None admin] PATH_INFO: `/v3/domains/60b0b84a843f49239f332c1a6ccff39c` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:37.191727 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0850aa-05c3-41eb-a0a6-1f77798be0e4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=60b0b84a843f49239f332c1a6ccff39c)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:37.193192 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0850aa-05c3-41eb-a0a6-1f77798be0e4 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:37.194740 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 586/1172] 10.222.0.45 () {56 vars in 1183 bytes} [Wed Aug 24 17:29:37 2022] GET /identity/v3/domains/60b0b84a843f49239f332c1a6ccff39c => generated 267 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:37.201008 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97029cd1-2106-44f5-b541-6f6584167703 None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:37.296391 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97029cd1-2106-44f5-b541-6f6584167703 None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:37.329137 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932c608f-320f-4f09-ad14-9fcb118a6b87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:37.329675 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932c608f-320f-4f09-ad14-9fcb118a6b87 None admin] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:37.329954 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932c608f-320f-4f09-ad14-9fcb118a6b87 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:37.330228 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932c608f-320f-4f09-ad14-9fcb118a6b87 None admin] PATH_INFO: `/v3/projects` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:37.330869 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932c608f-320f-4f09-ad14-9fcb118a6b87 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:37.332290 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932c608f-320f-4f09-ad14-9fcb118a6b87 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:37.341808 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:37.358954 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 587/1173] 10.222.0.45 () {60 vars in 1141 bytes} [Wed Aug 24 17:29:37 2022] POST /identity/v3/projects => generated 361 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:29:39.256333 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268367fb-e853-4f58-b19e-34c98c25e776 None None] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:39.256912 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268367fb-e853-4f58-b19e-34c98c25e776 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:39.256912 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268367fb-e853-4f58-b19e-34c98c25e776 None None] PATH_INFO: `/` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:39.258255 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 587/1174] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:29:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:29:39.264126 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42eac9a3-cd2a-4640-b87d-1cdca6c912e0 None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:39.264538 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42eac9a3-cd2a-4640-b87d-1cdca6c912e0 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:39.264698 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42eac9a3-cd2a-4640-b87d-1cdca6c912e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:39.282152 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-42eac9a3-cd2a-4640-b87d-1cdca6c912e0 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:29:39.283337 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-42eac9a3-cd2a-4640-b87d-1cdca6c912e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:29:39.347897 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 588/1175] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:29:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:29:39.617865 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268367fb-e853-4f58-b19e-34c98c25e776 None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:39.715175 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268367fb-e853-4f58-b19e-34c98c25e776 None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:39.748115 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eaf69f4-441a-4e73-9589-19874c0113fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:39.748753 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaf69f4-441a-4e73-9589-19874c0113fe None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:39.749033 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaf69f4-441a-4e73-9589-19874c0113fe None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:39.749304 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaf69f4-441a-4e73-9589-19874c0113fe None admin] PATH_INFO: `/v3/domains/60b0b84a843f49239f332c1a6ccff39c` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:39.750411 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eaf69f4-441a-4e73-9589-19874c0113fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=60b0b84a843f49239f332c1a6ccff39c)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:39.751825 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eaf69f4-441a-4e73-9589-19874c0113fe None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:39.753364 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 588/1176] 10.222.0.45 () {56 vars in 1183 bytes} [Wed Aug 24 17:29:39 2022] GET /identity/v3/domains/60b0b84a843f49239f332c1a6ccff39c => generated 267 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:39.758854 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42eac9a3-cd2a-4640-b87d-1cdca6c912e0 None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:39.854853 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42eac9a3-cd2a-4640-b87d-1cdca6c912e0 None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:39.874662 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:39.888957 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e944f8-7130-442f-9549-7e814d9ff806 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:39.889556 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e944f8-7130-442f-9549-7e814d9ff806 None admin] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:39.889830 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e944f8-7130-442f-9549-7e814d9ff806 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:39.890096 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e944f8-7130-442f-9549-7e814d9ff806 None admin] PATH_INFO: `/v3/users` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:39.890726 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e944f8-7130-442f-9549-7e814d9ff806 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:39.892139 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e944f8-7130-442f-9549-7e814d9ff806 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:39.913000 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 589/1177] 10.222.0.45 () {60 vars in 1132 bytes} [Wed Aug 24 17:29:39 2022] POST /identity/v3/users => generated 311 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:29:41.849883 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5377a144-3d03-4d98-bae9-c09bc815a4ce None None] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:41.850393 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5377a144-3d03-4d98-bae9-c09bc815a4ce None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:41.850492 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5377a144-3d03-4d98-bae9-c09bc815a4ce None None] PATH_INFO: `/` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:41.851776 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 589/1178] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:29:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:29:41.857549 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3560a50a-945a-4bc6-aaba-383845bcd464 None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:41.857922 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3560a50a-945a-4bc6-aaba-383845bcd464 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:41.858099 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3560a50a-945a-4bc6-aaba-383845bcd464 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:41.876467 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-3560a50a-945a-4bc6-aaba-383845bcd464 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:29:41.877680 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-3560a50a-945a-4bc6-aaba-383845bcd464 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:29:41.941785 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 590/1179] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:29:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:29:42.208633 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5377a144-3d03-4d98-bae9-c09bc815a4ce None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:42.263490 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:42.307079 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5377a144-3d03-4d98-bae9-c09bc815a4ce None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:42.339309 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28076d9-13db-4c22-b277-19fc57f210a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:42.340423 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28076d9-13db-4c22-b277-19fc57f210a5 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:42.340727 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28076d9-13db-4c22-b277-19fc57f210a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:42.340997 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28076d9-13db-4c22-b277-19fc57f210a5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:42.350640 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28076d9-13db-4c22-b277-19fc57f210a5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:29:42.355283 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application [None req-e28076d9-13db-4c22-b277-19fc57f210a5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:29:42.355283 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:29:42.355283 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:29:42.355283 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:29:42.355283 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:29:42.355283 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:29:42.355283 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:29:42.355283 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:29:42.355283 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:29:42.355283 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:29:42.355283 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:29:42.355283 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:29:42.355283 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:29:42.355283 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:29:42.355283 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:29:42.355283 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:29:42.357097 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:29:42.357097 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:29:42.357097 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:29:42.357097 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:29:42.357097 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:29:42.357097 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:29:42.357097 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:29:42.357097 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:29:42.357097 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:29:42.357097 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:29:42.357097 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:29:42.357097 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application with Lock( Aug 24 17:29:42.357097 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:29:42.357097 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:29:42.357097 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:29:42.357097 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:29:42.358299 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:29:42.358299 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:29:42.358299 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:29:42.358299 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:29:42.358299 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:29:42.358299 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:29:42.358299 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:29:42.358299 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:29:42.358299 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:29:42.358299 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:29:42.358299 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:29:42.358299 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Aug 24 17:29:42.358299 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 590/1180] 10.222.0.45 () {56 vars in 1096 bytes} [Wed Aug 24 17:29:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:29:42.363024 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3560a50a-945a-4bc6-aaba-383845bcd464 None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:42.456151 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3560a50a-945a-4bc6-aaba-383845bcd464 None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:42.463110 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:42.489982 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a96773-c1e1-469b-9379-0835fc513bb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:42.490575 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a96773-c1e1-469b-9379-0835fc513bb5 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:42.490848 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a96773-c1e1-469b-9379-0835fc513bb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:42.491117 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a96773-c1e1-469b-9379-0835fc513bb5 None admin] PATH_INFO: `/v3/roles` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:42.491768 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a96773-c1e1-469b-9379-0835fc513bb5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:29:42.492065 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a96773-c1e1-469b-9379-0835fc513bb5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:42.493453 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a96773-c1e1-469b-9379-0835fc513bb5 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:42.501081 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 591/1181] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Aug 24 17:29:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:42.506715 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28076d9-13db-4c22-b277-19fc57f210a5 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:42.602102 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28076d9-13db-4c22-b277-19fc57f210a5 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:42.615569 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:42.635036 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c2a96e-fcc8-4329-9ea3-c8a413371bea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:42.635641 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c2a96e-fcc8-4329-9ea3-c8a413371bea None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:42.635914 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c2a96e-fcc8-4329-9ea3-c8a413371bea None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:42.636181 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c2a96e-fcc8-4329-9ea3-c8a413371bea None admin] PATH_INFO: `/v3/users/85be1ca9d1af4205a2c408596f201f1c` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:42.649060 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c2a96e-fcc8-4329-9ea3-c8a413371bea None admin] RBAC: Authorizing `identity:get_user(user_id=85be1ca9d1af4205a2c408596f201f1c)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:42.650478 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c2a96e-fcc8-4329-9ea3-c8a413371bea None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:42.652022 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 591/1182] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Aug 24 17:29:42 2022] GET /identity/v3/users/85be1ca9d1af4205a2c408596f201f1c => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:42.658741 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a96773-c1e1-469b-9379-0835fc513bb5 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:42.754116 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a96773-c1e1-469b-9379-0835fc513bb5 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:42.787176 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6cbda7-92d1-455b-b8a6-d78e5112a802 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:42.787775 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6cbda7-92d1-455b-b8a6-d78e5112a802 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:42.788052 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6cbda7-92d1-455b-b8a6-d78e5112a802 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:42.788322 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6cbda7-92d1-455b-b8a6-d78e5112a802 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:42.821914 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:42.853986 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6cbda7-92d1-455b-b8a6-d78e5112a802 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:42.855499 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6cbda7-92d1-455b-b8a6-d78e5112a802 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:42.951829 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 592/1183] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Aug 24 17:29:42 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 17:29:42.957737 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c2a96e-fcc8-4329-9ea3-c8a413371bea None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:43.051641 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c2a96e-fcc8-4329-9ea3-c8a413371bea None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:43.084668 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451f9d53-b9ee-44fe-9c2c-be23738009ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:43.085379 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451f9d53-b9ee-44fe-9c2c-be23738009ac None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:43.085660 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451f9d53-b9ee-44fe-9c2c-be23738009ac None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:43.085933 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451f9d53-b9ee-44fe-9c2c-be23738009ac None admin] PATH_INFO: `/v3/projects/5e7f538f2bb84dbb94559196d1c07dc6` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:43.087029 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451f9d53-b9ee-44fe-9c2c-be23738009ac None admin] RBAC: Authorizing `identity:get_project(project_id=5e7f538f2bb84dbb94559196d1c07dc6)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:43.088440 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451f9d53-b9ee-44fe-9c2c-be23738009ac None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:43.090884 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 592/1184] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Aug 24 17:29:42 2022] GET /identity/v3/projects/5e7f538f2bb84dbb94559196d1c07dc6 => generated 361 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:43.096763 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6cbda7-92d1-455b-b8a6-d78e5112a802 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:43.161020 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:43.192662 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6cbda7-92d1-455b-b8a6-d78e5112a802 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:43.225963 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e96ed24-71d9-498d-b0da-12c23b7e4db0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:43.226549 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e96ed24-71d9-498d-b0da-12c23b7e4db0 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:43.226824 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e96ed24-71d9-498d-b0da-12c23b7e4db0 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:43.227093 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e96ed24-71d9-498d-b0da-12c23b7e4db0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:43.227806 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e96ed24-71d9-498d-b0da-12c23b7e4db0 None admin] RBAC: Adding query filter params (role.id=e411ad9eb96e4e8d85b59b5b700ada7b, scope.project.id=5e7f538f2bb84dbb94559196d1c07dc6, user.id=85be1ca9d1af4205a2c408596f201f1c) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:29:43.228111 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e96ed24-71d9-498d-b0da-12c23b7e4db0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:43.229555 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e96ed24-71d9-498d-b0da-12c23b7e4db0 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:43.235196 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 593/1185] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Aug 24 17:29:43 2022] GET /identity/v3/role_assignments?user.id=85be1ca9d1af4205a2c408596f201f1c&scope.project.id=5e7f538f2bb84dbb94559196d1c07dc6&role.id=e411ad9eb96e4e8d85b59b5b700ada7b => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:45.069380 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343aa7f8-5055-4f85-8914-45831cfbfa93 None None] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:45.069895 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343aa7f8-5055-4f85-8914-45831cfbfa93 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:45.069895 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343aa7f8-5055-4f85-8914-45831cfbfa93 None None] PATH_INFO: `/` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:45.071179 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 593/1186] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:29:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:29:45.076260 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd72933-0522-4bdf-9757-bc2a5f26503c None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:45.076725 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd72933-0522-4bdf-9757-bc2a5f26503c None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:45.076834 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd72933-0522-4bdf-9757-bc2a5f26503c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:45.094928 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-2cd72933-0522-4bdf-9757-bc2a5f26503c None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:29:45.096091 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-2cd72933-0522-4bdf-9757-bc2a5f26503c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:29:45.159249 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 594/1187] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:29:45 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:29:45.423123 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343aa7f8-5055-4f85-8914-45831cfbfa93 None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:45.464460 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:45.523416 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343aa7f8-5055-4f85-8914-45831cfbfa93 None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:45.555744 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9483aa5-5400-43d2-8915-37a20dd9d1f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:45.556367 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9483aa5-5400-43d2-8915-37a20dd9d1f5 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:45.556676 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9483aa5-5400-43d2-8915-37a20dd9d1f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:45.556952 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9483aa5-5400-43d2-8915-37a20dd9d1f5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:45.567184 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9483aa5-5400-43d2-8915-37a20dd9d1f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:29:45.571884 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application [None req-e9483aa5-5400-43d2-8915-37a20dd9d1f5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:29:45.571884 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:29:45.571884 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:29:45.571884 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:29:45.571884 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:29:45.571884 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:29:45.571884 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:29:45.571884 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:29:45.571884 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:29:45.571884 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:29:45.571884 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:29:45.571884 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:29:45.571884 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:29:45.571884 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:29:45.571884 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:29:45.571884 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:29:45.573012 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:29:45.573012 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:29:45.573012 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:29:45.573012 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:29:45.573012 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:29:45.573012 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:29:45.573012 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:29:45.573012 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:29:45.573012 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:29:45.573012 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:29:45.573012 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:29:45.573012 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application with Lock( Aug 24 17:29:45.573012 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:29:45.573012 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:29:45.573012 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:29:45.573012 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:29:45.574040 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:29:45.574040 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:29:45.574040 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:29:45.574040 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:29:45.574040 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:29:45.574040 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:29:45.574040 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:29:45.574040 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:29:45.574040 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:29:45.574040 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:29:45.574040 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:29:45.574040 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Aug 24 17:29:45.574040 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 594/1188] 10.222.0.45 () {56 vars in 1096 bytes} [Wed Aug 24 17:29:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:29:45.579054 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd72933-0522-4bdf-9757-bc2a5f26503c None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:45.637056 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:45.673803 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd72933-0522-4bdf-9757-bc2a5f26503c None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:45.705854 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1709f5-3264-4a8b-bfcc-c32ce9ca3d7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:45.706441 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1709f5-3264-4a8b-bfcc-c32ce9ca3d7c None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:45.706714 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1709f5-3264-4a8b-bfcc-c32ce9ca3d7c None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:45.706979 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1709f5-3264-4a8b-bfcc-c32ce9ca3d7c None admin] PATH_INFO: `/v3/roles` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:45.707627 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1709f5-3264-4a8b-bfcc-c32ce9ca3d7c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:29:45.707923 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1709f5-3264-4a8b-bfcc-c32ce9ca3d7c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:45.709290 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1709f5-3264-4a8b-bfcc-c32ce9ca3d7c None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:45.716902 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 595/1189] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Aug 24 17:29:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:45.722470 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9483aa5-5400-43d2-8915-37a20dd9d1f5 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:45.817008 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:45.820163 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9483aa5-5400-43d2-8915-37a20dd9d1f5 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:45.853055 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdc14fc-30f8-4c69-a13d-5f66b23c24ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:45.853670 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdc14fc-30f8-4c69-a13d-5f66b23c24ee None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:45.853949 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdc14fc-30f8-4c69-a13d-5f66b23c24ee None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:45.854217 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdc14fc-30f8-4c69-a13d-5f66b23c24ee None admin] PATH_INFO: `/v3/users/85be1ca9d1af4205a2c408596f201f1c` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:45.855323 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdc14fc-30f8-4c69-a13d-5f66b23c24ee None admin] RBAC: Authorizing `identity:get_user(user_id=85be1ca9d1af4205a2c408596f201f1c)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:45.857534 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdc14fc-30f8-4c69-a13d-5f66b23c24ee None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:45.859353 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 595/1190] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Aug 24 17:29:45 2022] GET /identity/v3/users/85be1ca9d1af4205a2c408596f201f1c => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:45.865351 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1709f5-3264-4a8b-bfcc-c32ce9ca3d7c None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:45.961147 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1709f5-3264-4a8b-bfcc-c32ce9ca3d7c None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:45.983588 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:45.994387 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2627a6c-4646-4355-b49d-b0f0686b7128 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:45.994996 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2627a6c-4646-4355-b49d-b0f0686b7128 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:45.995270 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2627a6c-4646-4355-b49d-b0f0686b7128 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:45.995538 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2627a6c-4646-4355-b49d-b0f0686b7128 None admin] PATH_INFO: `/v3/projects/5e7f538f2bb84dbb94559196d1c07dc6` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:45.996650 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2627a6c-4646-4355-b49d-b0f0686b7128 None admin] RBAC: Authorizing `identity:get_project(project_id=5e7f538f2bb84dbb94559196d1c07dc6)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:45.998063 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2627a6c-4646-4355-b49d-b0f0686b7128 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:45.999652 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 596/1191] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Aug 24 17:29:45 2022] GET /identity/v3/projects/5e7f538f2bb84dbb94559196d1c07dc6 => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:46.005461 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdc14fc-30f8-4c69-a13d-5f66b23c24ee None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:46.101379 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdc14fc-30f8-4c69-a13d-5f66b23c24ee None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:46.134178 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4710dd0b-5d1c-45cd-9412-33de2d3bff24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:46.134815 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4710dd0b-5d1c-45cd-9412-33de2d3bff24 None admin] REQUEST_METHOD: `PUT` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:46.135094 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4710dd0b-5d1c-45cd-9412-33de2d3bff24 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:46.135368 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4710dd0b-5d1c-45cd-9412-33de2d3bff24 None admin] PATH_INFO: `/v3/projects/5e7f538f2bb84dbb94559196d1c07dc6/users/85be1ca9d1af4205a2c408596f201f1c/roles/e411ad9eb96e4e8d85b59b5b700ada7b` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:46.137777 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4710dd0b-5d1c-45cd-9412-33de2d3bff24 None admin] RBAC: Authorizing `identity:create_grant(project_id=5e7f538f2bb84dbb94559196d1c07dc6, user_id=85be1ca9d1af4205a2c408596f201f1c, role_id=e411ad9eb96e4e8d85b59b5b700ada7b)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:46.139192 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4710dd0b-5d1c-45cd-9412-33de2d3bff24 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:46.151204 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 596/1192] 10.222.0.45 () {58 vars in 1439 bytes} [Wed Aug 24 17:29:46 2022] PUT /identity/v3/projects/5e7f538f2bb84dbb94559196d1c07dc6/users/85be1ca9d1af4205a2c408596f201f1c/roles/e411ad9eb96e4e8d85b59b5b700ada7b => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:29:48.030320 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927bc4d8-a4bd-4b74-adce-a3c286dff987 None None] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:48.030850 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927bc4d8-a4bd-4b74-adce-a3c286dff987 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:48.030850 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927bc4d8-a4bd-4b74-adce-a3c286dff987 None None] PATH_INFO: `/` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:48.032202 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 597/1193] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:29:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:29:48.037527 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dd05d1-725f-4144-b597-59ae7ca1f6ab None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:48.037933 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dd05d1-725f-4144-b597-59ae7ca1f6ab None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:48.038076 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dd05d1-725f-4144-b597-59ae7ca1f6ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:48.055574 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-e5dd05d1-725f-4144-b597-59ae7ca1f6ab None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:29:48.056728 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-e5dd05d1-725f-4144-b597-59ae7ca1f6ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:29:48.079565 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:48.122760 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 597/1194] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:29:48 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:29:48.394732 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927bc4d8-a4bd-4b74-adce-a3c286dff987 None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:48.494647 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927bc4d8-a4bd-4b74-adce-a3c286dff987 None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:48.527536 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e176ad41-117c-430c-b77d-bc120033fcaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:48.528160 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e176ad41-117c-430c-b77d-bc120033fcaf None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:48.528439 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e176ad41-117c-430c-b77d-bc120033fcaf None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:48.528747 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e176ad41-117c-430c-b77d-bc120033fcaf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:48.538831 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e176ad41-117c-430c-b77d-bc120033fcaf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:29:48.543961 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application [None req-e176ad41-117c-430c-b77d-bc120033fcaf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:29:48.543961 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:29:48.543961 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:29:48.543961 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:29:48.543961 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:29:48.543961 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:29:48.543961 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:29:48.543961 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:29:48.543961 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:29:48.543961 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:29:48.543961 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:29:48.543961 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:29:48.543961 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:29:48.543961 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:29:48.543961 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:29:48.543961 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:29:48.545134 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:29:48.545134 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:29:48.545134 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:29:48.545134 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:29:48.545134 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:29:48.545134 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:29:48.545134 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:29:48.545134 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:29:48.545134 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:29:48.545134 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:29:48.545134 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:29:48.545134 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application with Lock( Aug 24 17:29:48.545134 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:29:48.545134 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:29:48.545134 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:29:48.545134 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:29:48.546310 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:29:48.546310 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:29:48.546310 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:29:48.546310 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:29:48.546310 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:29:48.546310 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:29:48.546310 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:29:48.546310 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:29:48.546310 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:29:48.546310 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:29:48.546310 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:29:48.546310 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Aug 24 17:29:48.546310 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 598/1195] 10.222.0.45 () {56 vars in 1096 bytes} [Wed Aug 24 17:29:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:29:48.551036 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5dd05d1-725f-4144-b597-59ae7ca1f6ab None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:48.645833 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5dd05d1-725f-4144-b597-59ae7ca1f6ab None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:48.677722 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72d6329-1ffb-4155-bb88-931e9bf95aaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:48.678308 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72d6329-1ffb-4155-bb88-931e9bf95aaa None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:48.678584 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72d6329-1ffb-4155-bb88-931e9bf95aaa None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:48.678852 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72d6329-1ffb-4155-bb88-931e9bf95aaa None admin] PATH_INFO: `/v3/roles` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:48.679504 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72d6329-1ffb-4155-bb88-931e9bf95aaa None admin] RBAC: Adding query filter params (name=admin) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:29:48.679801 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72d6329-1ffb-4155-bb88-931e9bf95aaa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:48.681180 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72d6329-1ffb-4155-bb88-931e9bf95aaa None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:48.689308 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 598/1196] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Aug 24 17:29:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:48.694928 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e176ad41-117c-430c-b77d-bc120033fcaf None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:48.747024 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:48.792867 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e176ad41-117c-430c-b77d-bc120033fcaf None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:48.825684 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e461325-30bb-49a1-8814-0a03703b0868 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:48.826305 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e461325-30bb-49a1-8814-0a03703b0868 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:48.826588 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e461325-30bb-49a1-8814-0a03703b0868 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:48.826866 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e461325-30bb-49a1-8814-0a03703b0868 None admin] PATH_INFO: `/v3/users/85be1ca9d1af4205a2c408596f201f1c` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:48.827983 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e461325-30bb-49a1-8814-0a03703b0868 None admin] RBAC: Authorizing `identity:get_user(user_id=85be1ca9d1af4205a2c408596f201f1c)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:48.829441 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e461325-30bb-49a1-8814-0a03703b0868 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:48.831002 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 599/1197] 10.222.0.45 () {56 vars in 1177 bytes} [Wed Aug 24 17:29:48 2022] GET /identity/v3/users/85be1ca9d1af4205a2c408596f201f1c => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:48.836996 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72d6329-1ffb-4155-bb88-931e9bf95aaa None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:48.859957 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:48.935588 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72d6329-1ffb-4155-bb88-931e9bf95aaa None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:48.967591 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fb7f45-b5b8-48eb-a6a9-7ec0c2b5a143 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:48.968187 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fb7f45-b5b8-48eb-a6a9-7ec0c2b5a143 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:48.968468 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fb7f45-b5b8-48eb-a6a9-7ec0c2b5a143 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:48.968768 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fb7f45-b5b8-48eb-a6a9-7ec0c2b5a143 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:49.032760 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fb7f45-b5b8-48eb-a6a9-7ec0c2b5a143 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:49.034189 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fb7f45-b5b8-48eb-a6a9-7ec0c2b5a143 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:49.053280 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:49.128337 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 599/1198] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Aug 24 17:29:48 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 17:29:49.134226 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e461325-30bb-49a1-8814-0a03703b0868 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:49.230377 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e461325-30bb-49a1-8814-0a03703b0868 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:49.250006 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:49.263692 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66a44b2-904c-46cb-8d69-ca574950de3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:49.264314 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66a44b2-904c-46cb-8d69-ca574950de3d None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:49.264613 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66a44b2-904c-46cb-8d69-ca574950de3d None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:49.264887 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66a44b2-904c-46cb-8d69-ca574950de3d None admin] PATH_INFO: `/v3/projects/5e7f538f2bb84dbb94559196d1c07dc6` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:49.265990 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66a44b2-904c-46cb-8d69-ca574950de3d None admin] RBAC: Authorizing `identity:get_project(project_id=5e7f538f2bb84dbb94559196d1c07dc6)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:49.267411 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66a44b2-904c-46cb-8d69-ca574950de3d None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:49.269037 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 600/1199] 10.222.0.45 () {56 vars in 1186 bytes} [Wed Aug 24 17:29:49 2022] GET /identity/v3/projects/5e7f538f2bb84dbb94559196d1c07dc6 => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:49.275054 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fb7f45-b5b8-48eb-a6a9-7ec0c2b5a143 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:49.367745 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fb7f45-b5b8-48eb-a6a9-7ec0c2b5a143 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:49.387721 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:49.401248 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b58d499-9f03-4d0e-ad0d-7c400ba0db9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:49.401829 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b58d499-9f03-4d0e-ad0d-7c400ba0db9f None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:49.402103 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b58d499-9f03-4d0e-ad0d-7c400ba0db9f None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:49.402455 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b58d499-9f03-4d0e-ad0d-7c400ba0db9f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:49.403296 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b58d499-9f03-4d0e-ad0d-7c400ba0db9f None admin] RBAC: Adding query filter params (role.id=e411ad9eb96e4e8d85b59b5b700ada7b, scope.project.id=5e7f538f2bb84dbb94559196d1c07dc6, user.id=85be1ca9d1af4205a2c408596f201f1c) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:29:49.403597 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b58d499-9f03-4d0e-ad0d-7c400ba0db9f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:49.405065 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b58d499-9f03-4d0e-ad0d-7c400ba0db9f None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:49.410539 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 600/1200] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Aug 24 17:29:49 2022] GET /identity/v3/role_assignments?user.id=85be1ca9d1af4205a2c408596f201f1c&scope.project.id=5e7f538f2bb84dbb94559196d1c07dc6&role.id=e411ad9eb96e4e8d85b59b5b700ada7b => generated 608 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:59.259660 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=6 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:59.408008 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=8 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:44.610414 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e151ff-e195-47ff-b879-23295c248dac None None] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:44.611071 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e151ff-e195-47ff-b879-23295c248dac None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:44.611440 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e151ff-e195-47ff-b879-23295c248dac None None] PATH_INFO: `/` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:44.612767 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 601/1201] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:30:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:30:44.619037 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac68d9eb-1f90-428b-a413-a3b9616ebce4 None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:44.619501 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac68d9eb-1f90-428b-a413-a3b9616ebce4 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:44.619588 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac68d9eb-1f90-428b-a413-a3b9616ebce4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:44.643223 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-ac68d9eb-1f90-428b-a413-a3b9616ebce4 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:30:44.644763 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-ac68d9eb-1f90-428b-a413-a3b9616ebce4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:30:44.727866 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 601/1202] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:30:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:30:45.007791 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e151ff-e195-47ff-b879-23295c248dac None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:45.153441 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e151ff-e195-47ff-b879-23295c248dac None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:45.195215 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d88c6e-72df-420f-a51e-31ca6ba61549 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:45.195899 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d88c6e-72df-420f-a51e-31ca6ba61549 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:45.196204 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d88c6e-72df-420f-a51e-31ca6ba61549 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:45.196420 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d88c6e-72df-420f-a51e-31ca6ba61549 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:45.207937 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d88c6e-72df-420f-a51e-31ca6ba61549 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:45.210790 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d88c6e-72df-420f-a51e-31ca6ba61549 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:45.230907 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application [None req-d9d88c6e-72df-420f-a51e-31ca6ba61549 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:30:45.230907 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:30:45.230907 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:30:45.230907 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:30:45.230907 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:30:45.230907 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:30:45.230907 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:30:45.230907 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:30:45.230907 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:30:45.230907 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:30:45.230907 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:30:45.230907 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:30:45.230907 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:30:45.230907 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:30:45.230907 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:30:45.230907 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:30:45.232806 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:30:45.232806 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:30:45.232806 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:30:45.232806 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:30:45.232806 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:30:45.232806 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:30:45.232806 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:30:45.232806 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application with Lock( Aug 24 17:30:45.232806 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:30:45.232806 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:30:45.232806 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:30:45.232806 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:30:45.232806 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:30:45.232806 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:30:45.232806 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:30:45.232806 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:30:45.232806 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:30:45.234803 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:30:45.234803 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:30:45.234803 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Aug 24 17:30:45.234803 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 602/1203] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 17:30:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:30:45.241579 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac68d9eb-1f90-428b-a413-a3b9616ebce4 None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:45.377017 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac68d9eb-1f90-428b-a413-a3b9616ebce4 None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:45.418538 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:45.425994 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f86e27-1b69-44be-994e-87d8cb2bb858 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:45.426736 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f86e27-1b69-44be-994e-87d8cb2bb858 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:45.427063 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f86e27-1b69-44be-994e-87d8cb2bb858 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:45.427318 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f86e27-1b69-44be-994e-87d8cb2bb858 None admin] PATH_INFO: `/v3/domains` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:45.428123 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f86e27-1b69-44be-994e-87d8cb2bb858 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:30:45.428569 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f86e27-1b69-44be-994e-87d8cb2bb858 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:45.430166 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f86e27-1b69-44be-994e-87d8cb2bb858 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:45.453664 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 602/1204] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 24 17:30:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:30:45.455403 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d88c6e-72df-420f-a51e-31ca6ba61549 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:45.529632 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:45.598245 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d88c6e-72df-420f-a51e-31ca6ba61549 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:45.645797 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281f7dec-12ee-4f47-ad6d-93dfcc24e0b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:45.646450 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281f7dec-12ee-4f47-ad6d-93dfcc24e0b2 None admin] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:45.646693 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281f7dec-12ee-4f47-ad6d-93dfcc24e0b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:45.646988 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281f7dec-12ee-4f47-ad6d-93dfcc24e0b2 None admin] PATH_INFO: `/v3/users` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:45.647685 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281f7dec-12ee-4f47-ad6d-93dfcc24e0b2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:45.649325 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281f7dec-12ee-4f47-ad6d-93dfcc24e0b2 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:45.680623 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 603/1205] 10.222.0.45 () {60 vars in 1132 bytes} [Wed Aug 24 17:30:45 2022] POST /identity/v3/users => generated 251 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:30:47.871456 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0dff14-bcef-4b8e-84d7-c283c3b5d0c7 None None] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:47.872024 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0dff14-bcef-4b8e-84d7-c283c3b5d0c7 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:47.872024 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0dff14-bcef-4b8e-84d7-c283c3b5d0c7 None None] PATH_INFO: `/` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:47.873714 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 603/1206] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:30:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:30:47.879357 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad850cc-1ff6-4cd9-9a26-556842366453 None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:47.879969 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad850cc-1ff6-4cd9-9a26-556842366453 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:47.879969 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad850cc-1ff6-4cd9-9a26-556842366453 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:47.898976 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-fad850cc-1ff6-4cd9-9a26-556842366453 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:30:47.902804 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-fad850cc-1ff6-4cd9-9a26-556842366453 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:30:47.979902 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 604/1207] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:30:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 17:30:48.282627 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0dff14-bcef-4b8e-84d7-c283c3b5d0c7 None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:48.422654 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0dff14-bcef-4b8e-84d7-c283c3b5d0c7 None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:48.467612 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770c8672-bbb7-4be1-8a6d-f2cc2d44e40f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:48.468436 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770c8672-bbb7-4be1-8a6d-f2cc2d44e40f None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:48.468836 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770c8672-bbb7-4be1-8a6d-f2cc2d44e40f None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:48.469109 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770c8672-bbb7-4be1-8a6d-f2cc2d44e40f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:48.482159 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770c8672-bbb7-4be1-8a6d-f2cc2d44e40f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:30:48.487173 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application [None req-770c8672-bbb7-4be1-8a6d-f2cc2d44e40f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 17:30:48.487173 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:30:48.487173 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:30:48.487173 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:30:48.487173 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:30:48.487173 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:30:48.487173 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:30:48.487173 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:30:48.487173 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:30:48.487173 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:30:48.487173 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:30:48.487173 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:30:48.487173 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:30:48.487173 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:30:48.487173 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:30:48.487173 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:30:48.488179 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:30:48.488179 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:30:48.488179 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:30:48.488179 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:30:48.488179 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:30:48.488179 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:30:48.488179 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:30:48.488179 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:30:48.488179 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:30:48.488179 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:30:48.488179 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:30:48.488179 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application with Lock( Aug 24 17:30:48.488179 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:30:48.488179 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:30:48.488179 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:30:48.488179 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:30:48.489091 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:30:48.489091 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:30:48.489091 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:30:48.489091 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:30:48.489091 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:30:48.489091 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:30:48.489091 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:30:48.489091 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:30:48.489091 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:30:48.489091 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:30:48.489091 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 17:30:48.489091 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Aug 24 17:30:48.489091 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 604/1208] 10.222.0.45 () {56 vars in 1102 bytes} [Wed Aug 24 17:30:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:30:48.494895 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad850cc-1ff6-4cd9-9a26-556842366453 None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:48.566477 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:48.653400 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad850cc-1ff6-4cd9-9a26-556842366453 None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:48.699803 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554adbeb-c7cd-45f3-bc63-76f3f327af0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:48.700459 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554adbeb-c7cd-45f3-bc63-76f3f327af0f None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:48.700870 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554adbeb-c7cd-45f3-bc63-76f3f327af0f None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:48.701201 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554adbeb-c7cd-45f3-bc63-76f3f327af0f None admin] PATH_INFO: `/v3/roles` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:48.701939 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554adbeb-c7cd-45f3-bc63-76f3f327af0f None admin] RBAC: Adding query filter params (name=service) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:30:48.702320 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554adbeb-c7cd-45f3-bc63-76f3f327af0f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:48.703911 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554adbeb-c7cd-45f3-bc63-76f3f327af0f None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:48.719586 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 605/1209] 10.222.0.45 () {56 vars in 1103 bytes} [Wed Aug 24 17:30:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:30:48.721081 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770c8672-bbb7-4be1-8a6d-f2cc2d44e40f None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:48.796467 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:48.861897 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770c8672-bbb7-4be1-8a6d-f2cc2d44e40f None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:48.916640 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee82eae-c02e-4519-828c-2d43cb2fb1a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:48.918902 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee82eae-c02e-4519-828c-2d43cb2fb1a6 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:48.920209 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee82eae-c02e-4519-828c-2d43cb2fb1a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:48.920972 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee82eae-c02e-4519-828c-2d43cb2fb1a6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:48.938379 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee82eae-c02e-4519-828c-2d43cb2fb1a6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:48.939866 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee82eae-c02e-4519-828c-2d43cb2fb1a6 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:48.955301 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application [None req-2ee82eae-c02e-4519-828c-2d43cb2fb1a6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:30:48.955301 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:30:48.955301 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:30:48.955301 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:30:48.955301 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:30:48.955301 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:30:48.955301 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:30:48.955301 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:30:48.955301 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:30:48.955301 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:30:48.955301 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:30:48.955301 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:30:48.955301 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:30:48.955301 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:30:48.955301 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:30:48.955301 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:30:48.956787 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:30:48.956787 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:30:48.956787 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:30:48.956787 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:30:48.956787 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:30:48.956787 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:30:48.956787 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:30:48.956787 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application with Lock( Aug 24 17:30:48.956787 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:30:48.956787 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:30:48.956787 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:30:48.956787 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:30:48.956787 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:30:48.956787 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:30:48.956787 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:30:48.956787 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:30:48.956787 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:30:48.957710 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:30:48.957710 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:30:48.957710 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Aug 24 17:30:48.957710 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 605/1210] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 17:30:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:30:48.963412 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554adbeb-c7cd-45f3-bc63-76f3f327af0f None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:49.080708 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:49.085587 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554adbeb-c7cd-45f3-bc63-76f3f327af0f None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:49.140491 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc0ede5-eb11-41ab-a1bc-93211b1d705a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:49.141568 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc0ede5-eb11-41ab-a1bc-93211b1d705a None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:49.142072 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc0ede5-eb11-41ab-a1bc-93211b1d705a None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:49.142674 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc0ede5-eb11-41ab-a1bc-93211b1d705a None admin] PATH_INFO: `/v3/domains` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:49.143823 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc0ede5-eb11-41ab-a1bc-93211b1d705a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:30:49.144382 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc0ede5-eb11-41ab-a1bc-93211b1d705a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:49.146787 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc0ede5-eb11-41ab-a1bc-93211b1d705a None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:49.164572 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 606/1211] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 24 17:30:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:30:49.171325 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee82eae-c02e-4519-828c-2d43cb2fb1a6 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:49.320770 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee82eae-c02e-4519-828c-2d43cb2fb1a6 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:49.323226 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:49.375343 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bf0fbb-268a-4f00-b46e-e87e319c5c46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:49.376615 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bf0fbb-268a-4f00-b46e-e87e319c5c46 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:49.377328 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bf0fbb-268a-4f00-b46e-e87e319c5c46 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:49.378797 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bf0fbb-268a-4f00-b46e-e87e319c5c46 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:49.389860 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bf0fbb-268a-4f00-b46e-e87e319c5c46 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:49.391743 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bf0fbb-268a-4f00-b46e-e87e319c5c46 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:49.409325 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application [None req-96bf0fbb-268a-4f00-b46e-e87e319c5c46 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 17:30:49.409325 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:30:49.409325 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:30:49.409325 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:30:49.409325 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:30:49.409325 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:30:49.409325 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:30:49.409325 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:30:49.409325 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:30:49.409325 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:30:49.409325 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:30:49.409325 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:30:49.409325 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 17:30:49.409325 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 17:30:49.409325 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 17:30:49.409325 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 17:30:49.410520 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:30:49.410520 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:30:49.410520 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 17:30:49.410520 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:30:49.410520 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 17:30:49.410520 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:30:49.410520 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:30:49.410520 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:30:49.410520 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:30:49.410520 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:30:49.410520 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:30:49.410520 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application with Lock( Aug 24 17:30:49.410520 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:30:49.410520 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:30:49.410520 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:30:49.410520 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:30:49.410520 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:30:49.410520 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:30:49.411906 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:30:49.411906 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:30:49.411906 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 17:30:49.411906 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 17:30:49.411906 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 17:30:49.411906 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 17:30:49.411906 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 17:30:49.411906 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 17:30:49.411906 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 17:30:49.411906 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Aug 24 17:30:49.418278 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 606/1212] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 17:30:49 2022] GET /identity/v3/users/placement => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:30:49.418423 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc0ede5-eb11-41ab-a1bc-93211b1d705a None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:49.559144 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc0ede5-eb11-41ab-a1bc-93211b1d705a None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:49.609190 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3f6f3b-faa9-4159-bb67-283aeca5938e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:49.610092 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3f6f3b-faa9-4159-bb67-283aeca5938e None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:49.610598 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3f6f3b-faa9-4159-bb67-283aeca5938e None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:49.611015 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3f6f3b-faa9-4159-bb67-283aeca5938e None admin] PATH_INFO: `/v3/users` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:49.612194 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3f6f3b-faa9-4159-bb67-283aeca5938e None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:30:49.612586 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3f6f3b-faa9-4159-bb67-283aeca5938e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:49.614227 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3f6f3b-faa9-4159-bb67-283aeca5938e None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:49.642758 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 607/1213] 10.222.0.45 () {56 vars in 1143 bytes} [Wed Aug 24 17:30:49 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:30:49.646212 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bf0fbb-268a-4f00-b46e-e87e319c5c46 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:49.800577 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bf0fbb-268a-4f00-b46e-e87e319c5c46 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:49.848581 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fa3b01-1688-4d3c-8f4c-5d8be4722430 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:49.852597 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fa3b01-1688-4d3c-8f4c-5d8be4722430 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:49.852597 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fa3b01-1688-4d3c-8f4c-5d8be4722430 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:49.852597 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fa3b01-1688-4d3c-8f4c-5d8be4722430 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:49.886441 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:49.938999 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fa3b01-1688-4d3c-8f4c-5d8be4722430 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:49.940401 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fa3b01-1688-4d3c-8f4c-5d8be4722430 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:50.056451 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 607/1214] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Aug 24 17:30:49 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 17:30:50.062765 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3f6f3b-faa9-4159-bb67-283aeca5938e None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:50.093058 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:50.198145 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3f6f3b-faa9-4159-bb67-283aeca5938e None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:50.255169 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6698bfcb-c340-4c13-83a7-9b621c0a1d88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:50.255849 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6698bfcb-c340-4c13-83a7-9b621c0a1d88 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:50.256218 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6698bfcb-c340-4c13-83a7-9b621c0a1d88 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:50.256623 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6698bfcb-c340-4c13-83a7-9b621c0a1d88 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:50.269809 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6698bfcb-c340-4c13-83a7-9b621c0a1d88 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:50.271453 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6698bfcb-c340-4c13-83a7-9b621c0a1d88 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:50.290190 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application [None req-6698bfcb-c340-4c13-83a7-9b621c0a1d88 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:30:50.290190 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:30:50.290190 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:30:50.290190 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:30:50.290190 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:30:50.290190 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:30:50.290190 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:30:50.290190 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:30:50.290190 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:30:50.290190 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:30:50.290190 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:30:50.290190 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:30:50.290190 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:30:50.290190 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:30:50.290190 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:30:50.290190 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:30:50.291418 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:30:50.291418 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:30:50.291418 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:30:50.291418 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:30:50.291418 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:30:50.291418 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:30:50.291418 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:30:50.291418 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application with Lock( Aug 24 17:30:50.291418 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:30:50.291418 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:30:50.291418 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:30:50.291418 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:30:50.291418 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:30:50.291418 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:30:50.291418 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:30:50.291418 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:30:50.291418 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:30:50.292579 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:30:50.292579 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:30:50.292579 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Aug 24 17:30:50.297003 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 608/1215] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 17:30:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:30:50.298029 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fa3b01-1688-4d3c-8f4c-5d8be4722430 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:50.385464 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:50.437100 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fa3b01-1688-4d3c-8f4c-5d8be4722430 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:50.478950 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf874468-c2f2-445d-b22f-18116dd87cd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:50.479497 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf874468-c2f2-445d-b22f-18116dd87cd2 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:50.479775 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf874468-c2f2-445d-b22f-18116dd87cd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:50.480044 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf874468-c2f2-445d-b22f-18116dd87cd2 None admin] PATH_INFO: `/v3/domains` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:50.480714 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf874468-c2f2-445d-b22f-18116dd87cd2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:30:50.481067 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf874468-c2f2-445d-b22f-18116dd87cd2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:50.483039 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf874468-c2f2-445d-b22f-18116dd87cd2 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:50.497979 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 608/1216] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 24 17:30:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:30:50.503530 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6698bfcb-c340-4c13-83a7-9b621c0a1d88 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:50.604721 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:50.649178 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6698bfcb-c340-4c13-83a7-9b621c0a1d88 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:50.698250 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cb9e32-ed33-46f5-83bc-739a6702147f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:50.698962 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cb9e32-ed33-46f5-83bc-739a6702147f None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:50.699268 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cb9e32-ed33-46f5-83bc-739a6702147f None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:50.699471 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cb9e32-ed33-46f5-83bc-739a6702147f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:50.711682 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cb9e32-ed33-46f5-83bc-739a6702147f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:50.713579 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cb9e32-ed33-46f5-83bc-739a6702147f None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:50.727091 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application [None req-e1cb9e32-ed33-46f5-83bc-739a6702147f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:30:50.727091 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:30:50.727091 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:30:50.727091 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:30:50.727091 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:30:50.727091 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:30:50.727091 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:30:50.727091 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:30:50.727091 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:30:50.727091 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:30:50.727091 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:30:50.727091 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:30:50.727091 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 17:30:50.727091 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 17:30:50.727091 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 17:30:50.728175 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 17:30:50.728175 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:30:50.728175 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:30:50.728175 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:30:50.728175 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:30:50.728175 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:30:50.728175 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:30:50.728175 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:30:50.728175 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application with Lock( Aug 24 17:30:50.728175 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:30:50.728175 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:30:50.728175 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:30:50.728175 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:30:50.728175 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:30:50.728175 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:30:50.728175 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:30:50.728175 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:30:50.729797 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 17:30:50.729797 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 17:30:50.729797 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 17:30:50.729797 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 17:30:50.729797 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 17:30:50.729797 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 17:30:50.729797 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:30:50.729797 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Aug 24 17:30:50.729797 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 609/1217] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Aug 24 17:30:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:30:50.734841 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf874468-c2f2-445d-b22f-18116dd87cd2 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:50.883077 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf874468-c2f2-445d-b22f-18116dd87cd2 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:50.915287 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:50.931937 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38c77f1-0645-4135-b3b9-fca1475c6c7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:50.932827 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38c77f1-0645-4135-b3b9-fca1475c6c7a None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:50.933113 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38c77f1-0645-4135-b3b9-fca1475c6c7a None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:50.933384 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38c77f1-0645-4135-b3b9-fca1475c6c7a None admin] PATH_INFO: `/v3/projects` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:50.934084 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38c77f1-0645-4135-b3b9-fca1475c6c7a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:30:50.934390 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38c77f1-0645-4135-b3b9-fca1475c6c7a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:50.936274 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38c77f1-0645-4135-b3b9-fca1475c6c7a None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:50.952047 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 609/1218] 10.222.0.45 () {56 vars in 1148 bytes} [Wed Aug 24 17:30:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:30:50.958358 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cb9e32-ed33-46f5-83bc-739a6702147f None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:51.086192 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cb9e32-ed33-46f5-83bc-739a6702147f None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:51.121087 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:51.128132 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa98f63-5ffe-4e33-8d05-4c2ed4a04a50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:51.128766 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa98f63-5ffe-4e33-8d05-4c2ed4a04a50 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:51.129052 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa98f63-5ffe-4e33-8d05-4c2ed4a04a50 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:51.129401 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa98f63-5ffe-4e33-8d05-4c2ed4a04a50 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:51.130135 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa98f63-5ffe-4e33-8d05-4c2ed4a04a50 None admin] RBAC: Adding query filter params (role.id=ad6a81fb5d1941888577eb0ce9474882, scope.project.id=2ad8a32bd9d2434097bd08b111914ca1, user.id=37def4721b554b9ebb7704a19642d15b) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:30:51.130495 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa98f63-5ffe-4e33-8d05-4c2ed4a04a50 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:51.132109 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa98f63-5ffe-4e33-8d05-4c2ed4a04a50 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:51.140850 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 610/1219] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Aug 24 17:30:50 2022] GET /identity/v3/role_assignments?user.id=37def4721b554b9ebb7704a19642d15b&scope.project.id=2ad8a32bd9d2434097bd08b111914ca1&role.id=ad6a81fb5d1941888577eb0ce9474882 => generated 259 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:30:53.104899 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ccf2ab-a29a-45f3-a3df-0f05f0dda4d4 None None] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:53.105403 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ccf2ab-a29a-45f3-a3df-0f05f0dda4d4 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:53.105485 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ccf2ab-a29a-45f3-a3df-0f05f0dda4d4 None None] PATH_INFO: `/` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:53.107041 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 610/1220] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:30:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:30:53.113082 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2c6e87-56be-478b-9994-2194ba239c61 None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:53.113534 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2c6e87-56be-478b-9994-2194ba239c61 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:53.113687 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2c6e87-56be-478b-9994-2194ba239c61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:53.136272 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-eb2c6e87-56be-478b-9994-2194ba239c61 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:30:53.137539 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-eb2c6e87-56be-478b-9994-2194ba239c61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:30:53.227932 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 611/1221] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:30:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:30:53.506272 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ccf2ab-a29a-45f3-a3df-0f05f0dda4d4 None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:53.623438 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ccf2ab-a29a-45f3-a3df-0f05f0dda4d4 None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:53.663624 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06aa387b-83ce-4c36-81da-4645458b1f10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:53.664460 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06aa387b-83ce-4c36-81da-4645458b1f10 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:53.664920 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06aa387b-83ce-4c36-81da-4645458b1f10 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:53.665248 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06aa387b-83ce-4c36-81da-4645458b1f10 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:53.681146 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06aa387b-83ce-4c36-81da-4645458b1f10 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:30:53.686975 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application [None req-06aa387b-83ce-4c36-81da-4645458b1f10 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 17:30:53.686975 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:30:53.686975 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:30:53.686975 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:30:53.686975 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:30:53.686975 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:30:53.686975 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:30:53.686975 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:30:53.686975 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:30:53.686975 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:30:53.686975 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:30:53.686975 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:30:53.686975 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:30:53.686975 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:30:53.686975 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:30:53.686975 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:30:53.687958 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:30:53.687958 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:30:53.687958 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:30:53.687958 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:30:53.687958 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:30:53.687958 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:30:53.687958 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:30:53.687958 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:30:53.687958 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:30:53.687958 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:30:53.687958 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:30:53.687958 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application with Lock( Aug 24 17:30:53.687958 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:30:53.687958 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:30:53.687958 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:30:53.687958 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:30:53.688864 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:30:53.688864 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:30:53.688864 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:30:53.688864 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:30:53.688864 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:30:53.688864 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:30:53.688864 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:30:53.688864 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:30:53.688864 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:30:53.688864 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:30:53.688864 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 17:30:53.688864 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Aug 24 17:30:53.688864 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 611/1222] 10.222.0.45 () {56 vars in 1102 bytes} [Wed Aug 24 17:30:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:30:53.695035 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2c6e87-56be-478b-9994-2194ba239c61 None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:53.818701 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2c6e87-56be-478b-9994-2194ba239c61 None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:53.845964 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:53.862572 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2fbd99-0a40-4435-96ae-9d3313e071e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:53.863195 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2fbd99-0a40-4435-96ae-9d3313e071e8 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:53.863473 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2fbd99-0a40-4435-96ae-9d3313e071e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:53.863743 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2fbd99-0a40-4435-96ae-9d3313e071e8 None admin] PATH_INFO: `/v3/roles` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:53.864412 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2fbd99-0a40-4435-96ae-9d3313e071e8 None admin] RBAC: Adding query filter params (name=service) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:30:53.864766 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2fbd99-0a40-4435-96ae-9d3313e071e8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:53.866135 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2fbd99-0a40-4435-96ae-9d3313e071e8 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:53.875475 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 612/1223] 10.222.0.45 () {56 vars in 1103 bytes} [Wed Aug 24 17:30:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:30:53.881204 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06aa387b-83ce-4c36-81da-4645458b1f10 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:53.934395 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:54.002617 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06aa387b-83ce-4c36-81da-4645458b1f10 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:54.042953 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5beb8a31-6fff-44ba-83f9-cc64364283ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:54.043595 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5beb8a31-6fff-44ba-83f9-cc64364283ea None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:54.043875 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5beb8a31-6fff-44ba-83f9-cc64364283ea None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:54.044733 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5beb8a31-6fff-44ba-83f9-cc64364283ea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:54.055802 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5beb8a31-6fff-44ba-83f9-cc64364283ea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:54.057245 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5beb8a31-6fff-44ba-83f9-cc64364283ea None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:54.076733 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application [None req-5beb8a31-6fff-44ba-83f9-cc64364283ea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:30:54.076733 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:30:54.076733 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:30:54.076733 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:30:54.076733 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:30:54.076733 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:30:54.076733 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:30:54.076733 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:30:54.076733 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:30:54.076733 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:30:54.076733 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:30:54.076733 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:30:54.076733 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:30:54.076733 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:30:54.076733 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:30:54.076733 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:30:54.077933 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:30:54.077933 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:30:54.077933 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:30:54.077933 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:30:54.077933 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:30:54.077933 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:30:54.077933 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:30:54.077933 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application with Lock( Aug 24 17:30:54.077933 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:30:54.077933 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:30:54.077933 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:30:54.077933 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:30:54.077933 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:30:54.077933 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:30:54.077933 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:30:54.077933 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:30:54.077933 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:30:54.079053 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:30:54.079053 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:30:54.079053 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Aug 24 17:30:54.079909 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 612/1224] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 17:30:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:30:54.085906 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2fbd99-0a40-4435-96ae-9d3313e071e8 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:54.213085 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2fbd99-0a40-4435-96ae-9d3313e071e8 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:54.251718 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5167efd9-abfe-424f-9801-12d03ee4bf59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:54.252440 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5167efd9-abfe-424f-9801-12d03ee4bf59 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:54.252812 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5167efd9-abfe-424f-9801-12d03ee4bf59 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:54.253159 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5167efd9-abfe-424f-9801-12d03ee4bf59 None admin] PATH_INFO: `/v3/domains` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:54.253950 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5167efd9-abfe-424f-9801-12d03ee4bf59 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:30:54.254413 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5167efd9-abfe-424f-9801-12d03ee4bf59 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:54.256589 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5167efd9-abfe-424f-9801-12d03ee4bf59 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:54.270471 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 613/1225] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 24 17:30:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:30:54.276522 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5beb8a31-6fff-44ba-83f9-cc64364283ea None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:54.388459 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:54.397729 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5beb8a31-6fff-44ba-83f9-cc64364283ea None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:54.440739 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8922e72c-02bd-4154-bd7a-ab762459d0ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:54.441411 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8922e72c-02bd-4154-bd7a-ab762459d0ac None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:54.441698 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8922e72c-02bd-4154-bd7a-ab762459d0ac None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:54.441974 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8922e72c-02bd-4154-bd7a-ab762459d0ac None admin] PATH_INFO: `/v3/users/placement` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:54.451032 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8922e72c-02bd-4154-bd7a-ab762459d0ac None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:54.452495 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8922e72c-02bd-4154-bd7a-ab762459d0ac None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:54.465333 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application [None req-8922e72c-02bd-4154-bd7a-ab762459d0ac None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 17:30:54.465333 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:30:54.465333 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:30:54.465333 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:30:54.465333 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:30:54.465333 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:30:54.465333 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:30:54.465333 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:30:54.465333 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:30:54.465333 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:30:54.465333 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:30:54.465333 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:30:54.465333 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 17:30:54.465333 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 17:30:54.465333 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 17:30:54.465333 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 17:30:54.466473 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:30:54.466473 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:30:54.466473 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 17:30:54.466473 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:30:54.466473 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 17:30:54.466473 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:30:54.466473 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:30:54.466473 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:30:54.466473 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:30:54.466473 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:30:54.466473 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:30:54.466473 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application with Lock( Aug 24 17:30:54.466473 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:30:54.466473 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:30:54.466473 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:30:54.466473 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:30:54.466473 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:30:54.466473 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:30:54.467435 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:30:54.467435 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:30:54.467435 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 17:30:54.467435 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 17:30:54.467435 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 17:30:54.467435 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 17:30:54.467435 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 17:30:54.467435 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 17:30:54.467435 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 17:30:54.467435 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Aug 24 17:30:54.472600 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 613/1226] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 17:30:54 2022] GET /identity/v3/users/placement => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:30:54.473149 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5167efd9-abfe-424f-9801-12d03ee4bf59 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:54.538534 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:54.605063 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5167efd9-abfe-424f-9801-12d03ee4bf59 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:54.646438 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc8f81d-5e79-44a8-9823-94d0beacbc44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:54.647138 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc8f81d-5e79-44a8-9823-94d0beacbc44 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:54.647470 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc8f81d-5e79-44a8-9823-94d0beacbc44 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:54.647837 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc8f81d-5e79-44a8-9823-94d0beacbc44 None admin] PATH_INFO: `/v3/users` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:54.648627 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc8f81d-5e79-44a8-9823-94d0beacbc44 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:30:54.648944 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc8f81d-5e79-44a8-9823-94d0beacbc44 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:54.650458 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc8f81d-5e79-44a8-9823-94d0beacbc44 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:54.669916 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 614/1227] 10.222.0.45 () {56 vars in 1143 bytes} [Wed Aug 24 17:30:54 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:30:54.676304 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8922e72c-02bd-4154-bd7a-ab762459d0ac None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:54.809044 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8922e72c-02bd-4154-bd7a-ab762459d0ac None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:54.847599 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2783090d-504c-4a1b-9b2e-b75c4de09643 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:54.848317 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2783090d-504c-4a1b-9b2e-b75c4de09643 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:54.848775 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2783090d-504c-4a1b-9b2e-b75c4de09643 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:54.848917 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2783090d-504c-4a1b-9b2e-b75c4de09643 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:54.863150 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2783090d-504c-4a1b-9b2e-b75c4de09643 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:54.864757 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2783090d-504c-4a1b-9b2e-b75c4de09643 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:54.881821 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application [None req-2783090d-504c-4a1b-9b2e-b75c4de09643 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:30:54.881821 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:30:54.881821 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:30:54.881821 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:30:54.881821 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:30:54.881821 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:30:54.881821 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:30:54.881821 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:30:54.881821 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:30:54.881821 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:30:54.881821 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:30:54.881821 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:30:54.881821 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:30:54.881821 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:30:54.881821 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:30:54.881821 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:30:54.883050 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:30:54.883050 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:30:54.883050 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:30:54.883050 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:30:54.883050 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:30:54.883050 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:30:54.883050 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:30:54.883050 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application with Lock( Aug 24 17:30:54.883050 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:30:54.883050 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:30:54.883050 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:30:54.883050 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:30:54.883050 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:30:54.883050 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:30:54.883050 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:30:54.883050 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:30:54.883050 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:30:54.883943 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:30:54.883943 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:30:54.883943 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Aug 24 17:30:54.883943 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 614/1228] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 17:30:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:30:54.889818 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc8f81d-5e79-44a8-9823-94d0beacbc44 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:55.008015 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc8f81d-5e79-44a8-9823-94d0beacbc44 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:55.010856 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:55.049767 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6af8a05-566a-41ce-8506-caf7a4e9b73c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:55.050391 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6af8a05-566a-41ce-8506-caf7a4e9b73c None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:55.050677 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6af8a05-566a-41ce-8506-caf7a4e9b73c None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:55.050962 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6af8a05-566a-41ce-8506-caf7a4e9b73c None admin] PATH_INFO: `/v3/domains` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:55.051605 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6af8a05-566a-41ce-8506-caf7a4e9b73c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:30:55.051913 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6af8a05-566a-41ce-8506-caf7a4e9b73c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:55.053333 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6af8a05-566a-41ce-8506-caf7a4e9b73c None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:55.065902 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 615/1229] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 24 17:30:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:30:55.071844 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2783090d-504c-4a1b-9b2e-b75c4de09643 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:55.083150 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:55.194589 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2783090d-504c-4a1b-9b2e-b75c4de09643 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:55.235830 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d570dd2e-8644-46fa-891d-b50b2a9830ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:55.236458 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d570dd2e-8644-46fa-891d-b50b2a9830ee None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:55.237081 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d570dd2e-8644-46fa-891d-b50b2a9830ee None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:55.237405 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d570dd2e-8644-46fa-891d-b50b2a9830ee None admin] PATH_INFO: `/v3/projects/service` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:55.245092 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d570dd2e-8644-46fa-891d-b50b2a9830ee None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:55.246658 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d570dd2e-8644-46fa-891d-b50b2a9830ee None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:55.259849 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application [None req-d570dd2e-8644-46fa-891d-b50b2a9830ee None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:30:55.259849 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:30:55.259849 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:30:55.259849 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:30:55.259849 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:30:55.259849 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:30:55.259849 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:30:55.259849 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:30:55.259849 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:30:55.259849 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:30:55.259849 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:30:55.259849 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:30:55.259849 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 17:30:55.259849 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 17:30:55.259849 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 17:30:55.264887 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 17:30:55.264887 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:30:55.264887 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:30:55.264887 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:30:55.264887 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:30:55.264887 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:30:55.264887 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:30:55.264887 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:30:55.264887 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application with Lock( Aug 24 17:30:55.264887 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:30:55.264887 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:30:55.264887 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:30:55.264887 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:30:55.264887 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:30:55.264887 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:30:55.264887 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:30:55.264887 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:30:55.265817 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 17:30:55.265817 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 17:30:55.265817 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 17:30:55.265817 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 17:30:55.265817 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 17:30:55.265817 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 17:30:55.265817 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:30:55.265817 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Aug 24 17:30:55.265817 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 615/1230] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Aug 24 17:30:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:30:55.267058 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6af8a05-566a-41ce-8506-caf7a4e9b73c None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:55.398233 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6af8a05-566a-41ce-8506-caf7a4e9b73c None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:55.444088 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88478a1d-bad5-48cb-9526-9e88fd99f8a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:55.444737 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88478a1d-bad5-48cb-9526-9e88fd99f8a0 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:55.445406 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88478a1d-bad5-48cb-9526-9e88fd99f8a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:55.445678 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88478a1d-bad5-48cb-9526-9e88fd99f8a0 None admin] PATH_INFO: `/v3/projects` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:55.446383 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88478a1d-bad5-48cb-9526-9e88fd99f8a0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:30:55.446689 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88478a1d-bad5-48cb-9526-9e88fd99f8a0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:55.448130 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88478a1d-bad5-48cb-9526-9e88fd99f8a0 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:55.460953 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 616/1231] 10.222.0.45 () {56 vars in 1148 bytes} [Wed Aug 24 17:30:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:30:55.468076 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d570dd2e-8644-46fa-891d-b50b2a9830ee None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:55.542587 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:55.582800 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d570dd2e-8644-46fa-891d-b50b2a9830ee None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:55.625866 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5253286-c152-4fb5-87ef-9f116cdd8a45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:55.626755 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5253286-c152-4fb5-87ef-9f116cdd8a45 None admin] REQUEST_METHOD: `PUT` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:55.627046 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5253286-c152-4fb5-87ef-9f116cdd8a45 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:55.627338 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5253286-c152-4fb5-87ef-9f116cdd8a45 None admin] PATH_INFO: `/v3/projects/2ad8a32bd9d2434097bd08b111914ca1/users/37def4721b554b9ebb7704a19642d15b/roles/ad6a81fb5d1941888577eb0ce9474882` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:55.645065 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5253286-c152-4fb5-87ef-9f116cdd8a45 None admin] RBAC: Authorizing `identity:create_grant(project_id=2ad8a32bd9d2434097bd08b111914ca1, user_id=37def4721b554b9ebb7704a19642d15b, role_id=ad6a81fb5d1941888577eb0ce9474882)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:55.646477 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5253286-c152-4fb5-87ef-9f116cdd8a45 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:55.661364 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 616/1232] 10.222.0.45 () {58 vars in 1439 bytes} [Wed Aug 24 17:30:55 2022] PUT /identity/v3/projects/2ad8a32bd9d2434097bd08b111914ca1/users/37def4721b554b9ebb7704a19642d15b/roles/ad6a81fb5d1941888577eb0ce9474882 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:30:57.785619 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd46f53-489c-4b87-8f62-b50804326c03 None None] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:57.786150 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd46f53-489c-4b87-8f62-b50804326c03 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:57.786150 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd46f53-489c-4b87-8f62-b50804326c03 None None] PATH_INFO: `/` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:57.787510 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 617/1233] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:30:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:30:57.792843 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b026e89-edc1-4643-addd-e63c567e4246 None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:57.793256 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b026e89-edc1-4643-addd-e63c567e4246 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:57.793418 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b026e89-edc1-4643-addd-e63c567e4246 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:57.812607 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-2b026e89-edc1-4643-addd-e63c567e4246 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:30:57.813814 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-2b026e89-edc1-4643-addd-e63c567e4246 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:30:57.885677 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 617/1234] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:30:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:30:58.194395 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd46f53-489c-4b87-8f62-b50804326c03 None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:58.225695 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:58.315406 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd46f53-489c-4b87-8f62-b50804326c03 None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:58.354405 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf675134-05f5-46d4-b092-b9932f5f36a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:58.355035 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf675134-05f5-46d4-b092-b9932f5f36a8 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:58.355315 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf675134-05f5-46d4-b092-b9932f5f36a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:58.355591 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf675134-05f5-46d4-b092-b9932f5f36a8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:58.367568 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf675134-05f5-46d4-b092-b9932f5f36a8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:30:58.371984 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application [None req-cf675134-05f5-46d4-b092-b9932f5f36a8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 17:30:58.371984 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:30:58.371984 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:30:58.371984 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:30:58.371984 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:30:58.371984 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:30:58.371984 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:30:58.371984 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:30:58.371984 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:30:58.371984 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:30:58.371984 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:30:58.371984 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:30:58.371984 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:30:58.371984 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:30:58.371984 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:30:58.371984 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:30:58.373120 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:30:58.373120 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:30:58.373120 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:30:58.373120 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:30:58.373120 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:30:58.373120 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:30:58.373120 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:30:58.373120 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:30:58.373120 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:30:58.373120 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:30:58.373120 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:30:58.373120 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application with Lock( Aug 24 17:30:58.373120 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:30:58.373120 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:30:58.373120 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:30:58.373120 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:30:58.374064 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:30:58.374064 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:30:58.374064 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:30:58.374064 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:30:58.374064 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:30:58.374064 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:30:58.374064 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:30:58.374064 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:30:58.374064 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:30:58.374064 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:30:58.374064 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 17:30:58.374064 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Aug 24 17:30:58.374064 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 618/1235] 10.222.0.45 () {56 vars in 1102 bytes} [Wed Aug 24 17:30:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:30:58.379584 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b026e89-edc1-4643-addd-e63c567e4246 None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:58.431542 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:58.489344 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b026e89-edc1-4643-addd-e63c567e4246 None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:58.528580 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7342ef36-c624-416a-9a7f-9688e672a458 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:58.529343 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7342ef36-c624-416a-9a7f-9688e672a458 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:58.529631 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7342ef36-c624-416a-9a7f-9688e672a458 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:58.529912 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7342ef36-c624-416a-9a7f-9688e672a458 None admin] PATH_INFO: `/v3/roles` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:58.530590 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7342ef36-c624-416a-9a7f-9688e672a458 None admin] RBAC: Adding query filter params (name=service) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:30:58.530933 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7342ef36-c624-416a-9a7f-9688e672a458 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:58.532311 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7342ef36-c624-416a-9a7f-9688e672a458 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:58.542968 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 618/1236] 10.222.0.45 () {56 vars in 1103 bytes} [Wed Aug 24 17:30:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:30:58.548845 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf675134-05f5-46d4-b092-b9932f5f36a8 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:58.654532 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:58.676276 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf675134-05f5-46d4-b092-b9932f5f36a8 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:58.720828 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e2a28a-75d4-4cbf-90c6-c39f74ec3656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:58.721673 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e2a28a-75d4-4cbf-90c6-c39f74ec3656 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:58.721949 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e2a28a-75d4-4cbf-90c6-c39f74ec3656 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:58.722234 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e2a28a-75d4-4cbf-90c6-c39f74ec3656 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:58.739316 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e2a28a-75d4-4cbf-90c6-c39f74ec3656 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:58.740985 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e2a28a-75d4-4cbf-90c6-c39f74ec3656 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:58.757274 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application [None req-83e2a28a-75d4-4cbf-90c6-c39f74ec3656 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:30:58.757274 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:30:58.757274 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:30:58.757274 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:30:58.757274 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:30:58.757274 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:30:58.757274 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:30:58.757274 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:30:58.757274 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:30:58.757274 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:30:58.757274 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:30:58.757274 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:30:58.757274 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:30:58.757274 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:30:58.757274 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:30:58.757274 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:30:58.758772 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:30:58.758772 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:30:58.758772 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:30:58.758772 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:30:58.758772 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:30:58.758772 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:30:58.758772 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:30:58.758772 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application with Lock( Aug 24 17:30:58.758772 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:30:58.758772 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:30:58.758772 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:30:58.758772 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:30:58.758772 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:30:58.758772 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:30:58.758772 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:30:58.758772 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:30:58.758772 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:30:58.759709 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:30:58.759709 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:30:58.759709 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Aug 24 17:30:58.759709 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 619/1237] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 17:30:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:30:58.765527 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7342ef36-c624-416a-9a7f-9688e672a458 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:58.889371 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7342ef36-c624-416a-9a7f-9688e672a458 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:58.899880 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:58.931613 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7a2372-1434-4053-9d72-8c19d9c1df4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:58.932233 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7a2372-1434-4053-9d72-8c19d9c1df4a None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:58.932503 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7a2372-1434-4053-9d72-8c19d9c1df4a None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:58.932832 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7a2372-1434-4053-9d72-8c19d9c1df4a None admin] PATH_INFO: `/v3/domains` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:58.933437 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7a2372-1434-4053-9d72-8c19d9c1df4a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:30:58.933728 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7a2372-1434-4053-9d72-8c19d9c1df4a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:58.935700 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7a2372-1434-4053-9d72-8c19d9c1df4a None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:58.947881 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 619/1238] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 24 17:30:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:30:58.953830 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e2a28a-75d4-4cbf-90c6-c39f74ec3656 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:59.068974 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e2a28a-75d4-4cbf-90c6-c39f74ec3656 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:59.107332 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4004af4-838d-4833-afc3-06b10084a017 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:59.107968 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4004af4-838d-4833-afc3-06b10084a017 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:59.108249 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4004af4-838d-4833-afc3-06b10084a017 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:59.108526 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4004af4-838d-4833-afc3-06b10084a017 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:59.111258 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:59.118411 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4004af4-838d-4833-afc3-06b10084a017 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:59.119851 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4004af4-838d-4833-afc3-06b10084a017 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:59.131508 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application [None req-f4004af4-838d-4833-afc3-06b10084a017 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 17:30:59.131508 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:30:59.131508 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:30:59.131508 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:30:59.131508 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:30:59.131508 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:30:59.131508 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:30:59.131508 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:30:59.131508 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:30:59.131508 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:30:59.131508 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:30:59.131508 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:30:59.131508 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 17:30:59.131508 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 17:30:59.131508 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 17:30:59.131508 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 17:30:59.132649 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:30:59.132649 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:30:59.132649 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 17:30:59.132649 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:30:59.132649 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 17:30:59.132649 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:30:59.132649 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:30:59.132649 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:30:59.132649 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:30:59.132649 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:30:59.132649 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:30:59.132649 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application with Lock( Aug 24 17:30:59.132649 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:30:59.132649 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:30:59.132649 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:30:59.132649 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:30:59.132649 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:30:59.132649 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:30:59.133642 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:30:59.133642 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:30:59.133642 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 17:30:59.133642 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 17:30:59.133642 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 17:30:59.133642 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 17:30:59.133642 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 17:30:59.133642 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 17:30:59.133642 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 17:30:59.133642 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Aug 24 17:30:59.133642 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 620/1239] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 17:30:58 2022] GET /identity/v3/users/placement => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:30:59.138781 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7a2372-1434-4053-9d72-8c19d9c1df4a None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:59.250924 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7a2372-1434-4053-9d72-8c19d9c1df4a None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:59.287663 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7080a6ed-dfd9-4523-ba38-e9d7a2a54b51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:59.288304 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7080a6ed-dfd9-4523-ba38-e9d7a2a54b51 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:59.288645 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7080a6ed-dfd9-4523-ba38-e9d7a2a54b51 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:59.288947 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7080a6ed-dfd9-4523-ba38-e9d7a2a54b51 None admin] PATH_INFO: `/v3/users` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:59.289768 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7080a6ed-dfd9-4523-ba38-e9d7a2a54b51 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:30:59.290085 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7080a6ed-dfd9-4523-ba38-e9d7a2a54b51 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:59.291607 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7080a6ed-dfd9-4523-ba38-e9d7a2a54b51 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:59.310426 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 620/1240] 10.222.0.45 () {56 vars in 1143 bytes} [Wed Aug 24 17:30:59 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:30:59.316504 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4004af4-838d-4833-afc3-06b10084a017 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:59.426367 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4004af4-838d-4833-afc3-06b10084a017 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:59.463729 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe70f67-20e3-4ded-8e01-640b85c5eb82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:59.464468 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe70f67-20e3-4ded-8e01-640b85c5eb82 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:59.464770 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe70f67-20e3-4ded-8e01-640b85c5eb82 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:59.465116 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe70f67-20e3-4ded-8e01-640b85c5eb82 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:59.540235 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:59.547985 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe70f67-20e3-4ded-8e01-640b85c5eb82 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:59.549464 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe70f67-20e3-4ded-8e01-640b85c5eb82 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:59.658326 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 621/1241] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Aug 24 17:30:59 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 24 17:30:59.664286 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7080a6ed-dfd9-4523-ba38-e9d7a2a54b51 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:59.693077 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:59.771106 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7080a6ed-dfd9-4523-ba38-e9d7a2a54b51 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:59.809866 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c08739-bd3a-4112-a62f-12ede534642e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:59.810511 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c08739-bd3a-4112-a62f-12ede534642e None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:59.810778 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c08739-bd3a-4112-a62f-12ede534642e None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:59.811046 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c08739-bd3a-4112-a62f-12ede534642e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:59.821971 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c08739-bd3a-4112-a62f-12ede534642e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:59.823873 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c08739-bd3a-4112-a62f-12ede534642e None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:59.842225 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application [None req-32c08739-bd3a-4112-a62f-12ede534642e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:30:59.842225 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:30:59.842225 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:30:59.842225 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:30:59.842225 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:30:59.842225 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:30:59.842225 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:30:59.842225 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:30:59.842225 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:30:59.842225 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:30:59.842225 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:30:59.842225 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:30:59.842225 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:30:59.842225 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:30:59.842225 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:30:59.842225 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:30:59.843439 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:30:59.843439 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:30:59.843439 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:30:59.843439 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:30:59.843439 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:30:59.843439 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:30:59.843439 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:30:59.843439 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application with Lock( Aug 24 17:30:59.843439 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:30:59.843439 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:30:59.843439 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:30:59.843439 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:30:59.843439 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:30:59.843439 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:30:59.843439 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:30:59.843439 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:30:59.843439 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:30:59.844321 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:30:59.844321 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:30:59.844321 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Aug 24 17:30:59.844321 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 621/1242] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 17:30:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:30:59.849649 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe70f67-20e3-4ded-8e01-640b85c5eb82 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:59.948913 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:59.956411 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe70f67-20e3-4ded-8e01-640b85c5eb82 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:59.994008 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663bf7f3-8e49-48cc-a582-1efd20255ac5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:59.994778 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663bf7f3-8e49-48cc-a582-1efd20255ac5 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:59.995163 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663bf7f3-8e49-48cc-a582-1efd20255ac5 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:59.995535 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663bf7f3-8e49-48cc-a582-1efd20255ac5 None admin] PATH_INFO: `/v3/domains` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:59.996186 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663bf7f3-8e49-48cc-a582-1efd20255ac5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:30:59.996493 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663bf7f3-8e49-48cc-a582-1efd20255ac5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:59.998486 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663bf7f3-8e49-48cc-a582-1efd20255ac5 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:31:00.011668 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 622/1243] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 24 17:30:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:31:00.018304 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c08739-bd3a-4112-a62f-12ede534642e None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:31:00.103996 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:31:00.129774 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c08739-bd3a-4112-a62f-12ede534642e None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:31:00.167104 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180f58f9-9859-4771-9d40-1541ded94e26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:31:00.167811 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180f58f9-9859-4771-9d40-1541ded94e26 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:00.168140 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180f58f9-9859-4771-9d40-1541ded94e26 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:00.168475 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180f58f9-9859-4771-9d40-1541ded94e26 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:00.176478 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180f58f9-9859-4771-9d40-1541ded94e26 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:31:00.178164 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180f58f9-9859-4771-9d40-1541ded94e26 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:31:00.191771 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application [None req-180f58f9-9859-4771-9d40-1541ded94e26 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:31:00.191771 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:31:00.191771 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:31:00.191771 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:31:00.191771 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:31:00.191771 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:31:00.191771 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:31:00.191771 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:31:00.191771 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:31:00.191771 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:31:00.191771 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:31:00.191771 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:31:00.191771 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 17:31:00.191771 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 17:31:00.191771 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 17:31:00.193161 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 17:31:00.193161 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:31:00.193161 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:31:00.193161 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:31:00.193161 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:31:00.193161 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:31:00.193161 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:31:00.193161 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:31:00.193161 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application with Lock( Aug 24 17:31:00.193161 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:31:00.193161 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:31:00.193161 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:31:00.193161 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:31:00.193161 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:31:00.193161 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:31:00.193161 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:31:00.193161 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:31:00.194180 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 17:31:00.194180 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 17:31:00.194180 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 17:31:00.194180 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 17:31:00.194180 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 17:31:00.194180 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 17:31:00.194180 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:31:00.194180 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Aug 24 17:31:00.194180 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 622/1244] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Aug 24 17:31:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:31:00.200142 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663bf7f3-8e49-48cc-a582-1efd20255ac5 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:31:00.320645 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663bf7f3-8e49-48cc-a582-1efd20255ac5 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:31:00.358373 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7559bb2-41c5-42bf-b65a-dd60ab282d4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:31:00.359116 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7559bb2-41c5-42bf-b65a-dd60ab282d4d None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:00.359397 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7559bb2-41c5-42bf-b65a-dd60ab282d4d None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:00.359671 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7559bb2-41c5-42bf-b65a-dd60ab282d4d None admin] PATH_INFO: `/v3/projects` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:00.360367 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7559bb2-41c5-42bf-b65a-dd60ab282d4d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:31:00.360712 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7559bb2-41c5-42bf-b65a-dd60ab282d4d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:31:00.362832 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7559bb2-41c5-42bf-b65a-dd60ab282d4d None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:31:00.378995 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 623/1245] 10.222.0.45 () {56 vars in 1148 bytes} [Wed Aug 24 17:31:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:31:00.385056 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180f58f9-9859-4771-9d40-1541ded94e26 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:31:00.492840 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180f58f9-9859-4771-9d40-1541ded94e26 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:31:00.531659 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bef657d-149d-4f56-9a76-90dea9f28901 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:31:00.532722 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bef657d-149d-4f56-9a76-90dea9f28901 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:00.533211 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bef657d-149d-4f56-9a76-90dea9f28901 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:00.533696 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bef657d-149d-4f56-9a76-90dea9f28901 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:00.535017 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bef657d-149d-4f56-9a76-90dea9f28901 None admin] RBAC: Adding query filter params (role.id=ad6a81fb5d1941888577eb0ce9474882, scope.project.id=2ad8a32bd9d2434097bd08b111914ca1, user.id=37def4721b554b9ebb7704a19642d15b) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:31:00.535550 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bef657d-149d-4f56-9a76-90dea9f28901 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:31:00.538203 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bef657d-149d-4f56-9a76-90dea9f28901 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:31:00.547112 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 623/1246] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Aug 24 17:31:00 2022] GET /identity/v3/role_assignments?user.id=37def4721b554b9ebb7704a19642d15b&scope.project.id=2ad8a32bd9d2434097bd08b111914ca1&role.id=ad6a81fb5d1941888577eb0ce9474882 => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:31:02.627687 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7678940-d902-439c-9be2-aefcd930dfe0 None None] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:02.628258 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7678940-d902-439c-9be2-aefcd930dfe0 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:02.628258 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7678940-d902-439c-9be2-aefcd930dfe0 None None] PATH_INFO: `/` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:02.629810 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 624/1247] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:31:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:31:02.636012 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78988a73-50a2-4559-9acb-d555c07ed6c3 None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:02.636449 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78988a73-50a2-4559-9acb-d555c07ed6c3 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:02.636595 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78988a73-50a2-4559-9acb-d555c07ed6c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:02.644278 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:31:02.659016 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-78988a73-50a2-4559-9acb-d555c07ed6c3 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:31:02.660488 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-78988a73-50a2-4559-9acb-d555c07ed6c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:31:02.738280 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 624/1248] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:31:02 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:31:03.025013 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7678940-d902-439c-9be2-aefcd930dfe0 None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:31:03.045487 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:31:03.147650 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7678940-d902-439c-9be2-aefcd930dfe0 None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:31:03.188579 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e9941d-9532-41f0-a86e-09dbc25115e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:31:03.189342 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e9941d-9532-41f0-a86e-09dbc25115e8 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:03.189614 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e9941d-9532-41f0-a86e-09dbc25115e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:03.189887 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e9941d-9532-41f0-a86e-09dbc25115e8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:03.201858 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e9941d-9532-41f0-a86e-09dbc25115e8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:31:03.207301 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application [None req-f4e9941d-9532-41f0-a86e-09dbc25115e8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:31:03.207301 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:31:03.207301 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:31:03.207301 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:31:03.207301 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:31:03.207301 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:31:03.207301 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:31:03.207301 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:31:03.207301 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:31:03.207301 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:31:03.207301 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:31:03.207301 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:31:03.207301 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:31:03.207301 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:31:03.207301 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:31:03.207301 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:31:03.208301 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:31:03.208301 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:31:03.208301 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:31:03.208301 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:31:03.208301 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:31:03.208301 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:31:03.208301 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:31:03.208301 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:31:03.208301 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:31:03.208301 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:31:03.208301 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:31:03.208301 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application with Lock( Aug 24 17:31:03.208301 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:31:03.208301 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:31:03.208301 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:31:03.208301 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:31:03.209235 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:31:03.209235 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:31:03.209235 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:31:03.209235 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:31:03.209235 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:31:03.209235 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:31:03.209235 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:31:03.209235 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:31:03.209235 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:31:03.209235 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:31:03.209235 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:31:03.209235 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Aug 24 17:31:03.209235 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 625/1249] 10.222.0.45 () {56 vars in 1096 bytes} [Wed Aug 24 17:31:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:31:03.215530 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78988a73-50a2-4559-9acb-d555c07ed6c3 None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:31:03.334142 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78988a73-50a2-4559-9acb-d555c07ed6c3 None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:31:03.365631 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:31:03.371205 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e113010-b297-452f-9b64-864795d08e68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:31:03.371808 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e113010-b297-452f-9b64-864795d08e68 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:03.372086 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e113010-b297-452f-9b64-864795d08e68 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:03.372920 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e113010-b297-452f-9b64-864795d08e68 None admin] PATH_INFO: `/v3/roles` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:03.373578 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e113010-b297-452f-9b64-864795d08e68 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:31:03.373875 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e113010-b297-452f-9b64-864795d08e68 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:31:03.375227 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e113010-b297-452f-9b64-864795d08e68 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:31:03.383165 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 625/1250] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Aug 24 17:31:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:31:03.390595 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e9941d-9532-41f0-a86e-09dbc25115e8 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:31:03.474285 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:31:03.502153 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e9941d-9532-41f0-a86e-09dbc25115e8 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:31:03.538487 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92323810-28d8-49ca-bba5-f535691dfb87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:31:03.539121 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92323810-28d8-49ca-bba5-f535691dfb87 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:03.539402 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92323810-28d8-49ca-bba5-f535691dfb87 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:03.539672 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92323810-28d8-49ca-bba5-f535691dfb87 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:03.550477 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92323810-28d8-49ca-bba5-f535691dfb87 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:31:03.551924 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92323810-28d8-49ca-bba5-f535691dfb87 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:31:03.566289 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application [None req-92323810-28d8-49ca-bba5-f535691dfb87 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:31:03.566289 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:31:03.566289 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:31:03.566289 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:31:03.566289 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:31:03.566289 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:31:03.566289 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:31:03.566289 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:31:03.566289 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:31:03.566289 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:31:03.566289 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:31:03.566289 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:31:03.566289 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:31:03.566289 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:31:03.566289 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:31:03.566289 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:31:03.567388 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:31:03.567388 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:31:03.567388 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:31:03.567388 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:31:03.567388 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:31:03.567388 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:31:03.567388 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:31:03.567388 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application with Lock( Aug 24 17:31:03.567388 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:31:03.567388 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:31:03.567388 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:31:03.567388 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:31:03.567388 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:31:03.567388 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:31:03.567388 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:31:03.567388 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:31:03.567388 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:31:03.568288 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:31:03.568288 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:31:03.568288 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Aug 24 17:31:03.568288 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 626/1251] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 17:31:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:31:03.574035 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e113010-b297-452f-9b64-864795d08e68 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:31:03.688144 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e113010-b297-452f-9b64-864795d08e68 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:31:03.722391 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f89488-b5d2-46f6-8740-af695a4e7499 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:31:03.723090 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f89488-b5d2-46f6-8740-af695a4e7499 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:03.723445 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f89488-b5d2-46f6-8740-af695a4e7499 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:03.723808 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f89488-b5d2-46f6-8740-af695a4e7499 None admin] PATH_INFO: `/v3/domains` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:03.724519 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f89488-b5d2-46f6-8740-af695a4e7499 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:31:03.724816 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f89488-b5d2-46f6-8740-af695a4e7499 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:31:03.726151 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f89488-b5d2-46f6-8740-af695a4e7499 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:31:03.746444 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 626/1252] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 24 17:31:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:31:03.748269 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92323810-28d8-49ca-bba5-f535691dfb87 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:31:03.865700 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92323810-28d8-49ca-bba5-f535691dfb87 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:31:03.894731 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:31:03.904090 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac1496b-1a3f-4953-bb05-003a0b64e03e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:31:03.904763 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac1496b-1a3f-4953-bb05-003a0b64e03e None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:03.905045 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac1496b-1a3f-4953-bb05-003a0b64e03e None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:03.905321 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac1496b-1a3f-4953-bb05-003a0b64e03e None admin] PATH_INFO: `/v3/users/placement` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:03.914228 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac1496b-1a3f-4953-bb05-003a0b64e03e None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:31:03.916029 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac1496b-1a3f-4953-bb05-003a0b64e03e None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:31:03.929533 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application [None req-aac1496b-1a3f-4953-bb05-003a0b64e03e None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 17:31:03.929533 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:31:03.929533 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:31:03.929533 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:31:03.929533 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:31:03.929533 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:31:03.929533 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:31:03.929533 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:31:03.929533 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:31:03.929533 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:31:03.929533 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:31:03.929533 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:31:03.929533 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 17:31:03.929533 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 17:31:03.929533 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 17:31:03.929533 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 17:31:03.930685 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:31:03.930685 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:31:03.930685 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 17:31:03.930685 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:31:03.930685 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 17:31:03.930685 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:31:03.930685 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:31:03.930685 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:31:03.930685 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:31:03.930685 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:31:03.930685 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:31:03.930685 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application with Lock( Aug 24 17:31:03.930685 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:31:03.930685 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:31:03.930685 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:31:03.930685 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:31:03.930685 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:31:03.930685 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:31:03.931742 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:31:03.931742 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:31:03.931742 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 17:31:03.931742 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 17:31:03.931742 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 17:31:03.931742 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 17:31:03.931742 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 17:31:03.931742 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 17:31:03.931742 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 17:31:03.931742 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Aug 24 17:31:03.931742 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 627/1253] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 17:31:03 2022] GET /identity/v3/users/placement => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:31:03.937867 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f89488-b5d2-46f6-8740-af695a4e7499 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:31:04.001940 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:31:04.048806 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f89488-b5d2-46f6-8740-af695a4e7499 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:31:04.084765 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6239a437-479b-4325-ae2d-0396c71f9f99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:31:04.085378 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6239a437-479b-4325-ae2d-0396c71f9f99 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:04.085729 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6239a437-479b-4325-ae2d-0396c71f9f99 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:04.085933 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6239a437-479b-4325-ae2d-0396c71f9f99 None admin] PATH_INFO: `/v3/users` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:04.086664 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6239a437-479b-4325-ae2d-0396c71f9f99 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:31:04.086972 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6239a437-479b-4325-ae2d-0396c71f9f99 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:31:04.088466 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6239a437-479b-4325-ae2d-0396c71f9f99 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:31:04.107460 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 627/1254] 10.222.0.45 () {56 vars in 1143 bytes} [Wed Aug 24 17:31:03 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:31:04.113302 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac1496b-1a3f-4953-bb05-003a0b64e03e None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:31:04.238938 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac1496b-1a3f-4953-bb05-003a0b64e03e None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:31:04.272137 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784a7141-9228-42b4-bef1-e8e16ffc4ab9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:31:04.272711 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784a7141-9228-42b4-bef1-e8e16ffc4ab9 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:04.273007 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784a7141-9228-42b4-bef1-e8e16ffc4ab9 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:04.273268 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784a7141-9228-42b4-bef1-e8e16ffc4ab9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:04.330846 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:31:04.347438 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784a7141-9228-42b4-bef1-e8e16ffc4ab9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:31:04.348922 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784a7141-9228-42b4-bef1-e8e16ffc4ab9 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:31:04.456110 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 628/1255] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Aug 24 17:31:04 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 17:31:04.462165 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6239a437-479b-4325-ae2d-0396c71f9f99 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:31:04.569129 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6239a437-479b-4325-ae2d-0396c71f9f99 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:31:04.583443 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:31:04.605960 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48de66b-13ba-4cdd-bf45-2c950932f780 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:31:04.607167 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48de66b-13ba-4cdd-bf45-2c950932f780 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:04.607410 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48de66b-13ba-4cdd-bf45-2c950932f780 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:04.607711 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48de66b-13ba-4cdd-bf45-2c950932f780 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:04.617918 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48de66b-13ba-4cdd-bf45-2c950932f780 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:31:04.619402 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48de66b-13ba-4cdd-bf45-2c950932f780 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:31:04.633413 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application [None req-a48de66b-13ba-4cdd-bf45-2c950932f780 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:31:04.633413 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:31:04.633413 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:31:04.633413 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:31:04.633413 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:31:04.633413 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:31:04.633413 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:31:04.633413 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:31:04.633413 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:31:04.633413 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:31:04.633413 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:31:04.633413 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:31:04.633413 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:31:04.633413 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:31:04.633413 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:31:04.633413 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:31:04.634558 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:31:04.634558 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:31:04.634558 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:31:04.634558 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:31:04.634558 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:31:04.634558 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:31:04.634558 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:31:04.634558 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application with Lock( Aug 24 17:31:04.634558 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:31:04.634558 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:31:04.634558 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:31:04.634558 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:31:04.634558 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:31:04.634558 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:31:04.634558 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:31:04.634558 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:31:04.634558 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:31:04.635667 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:31:04.635667 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:31:04.635667 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Aug 24 17:31:04.639291 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 628/1256] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 17:31:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:31:04.640724 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784a7141-9228-42b4-bef1-e8e16ffc4ab9 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:31:04.732926 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:31:04.745809 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784a7141-9228-42b4-bef1-e8e16ffc4ab9 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:31:04.782860 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e9236f-4a43-432f-a4c7-32dd47321e0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:31:04.783533 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e9236f-4a43-432f-a4c7-32dd47321e0d None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:04.783802 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e9236f-4a43-432f-a4c7-32dd47321e0d None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:04.784020 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e9236f-4a43-432f-a4c7-32dd47321e0d None admin] PATH_INFO: `/v3/domains` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:04.784705 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e9236f-4a43-432f-a4c7-32dd47321e0d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:31:04.785012 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e9236f-4a43-432f-a4c7-32dd47321e0d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:31:04.786395 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e9236f-4a43-432f-a4c7-32dd47321e0d None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:31:04.802193 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 629/1257] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 24 17:31:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:31:04.808016 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48de66b-13ba-4cdd-bf45-2c950932f780 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:31:04.919165 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48de66b-13ba-4cdd-bf45-2c950932f780 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:31:04.955010 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6688f72-e336-4a9d-997e-8f3f356b024d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:31:04.955727 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6688f72-e336-4a9d-997e-8f3f356b024d None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:04.956098 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6688f72-e336-4a9d-997e-8f3f356b024d None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:04.956429 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6688f72-e336-4a9d-997e-8f3f356b024d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:04.964152 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6688f72-e336-4a9d-997e-8f3f356b024d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:31:04.965797 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6688f72-e336-4a9d-997e-8f3f356b024d None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:31:04.978785 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application [None req-d6688f72-e336-4a9d-997e-8f3f356b024d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:31:04.978785 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:31:04.978785 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:31:04.978785 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:31:04.978785 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:31:04.978785 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:31:04.978785 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:31:04.978785 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:31:04.978785 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:31:04.978785 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:31:04.978785 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:31:04.978785 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:31:04.978785 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 17:31:04.978785 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 17:31:04.978785 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 17:31:04.979934 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 17:31:04.979934 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:31:04.979934 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:31:04.979934 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:31:04.979934 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:31:04.979934 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:31:04.979934 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:31:04.979934 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:31:04.979934 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application with Lock( Aug 24 17:31:04.979934 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:31:04.979934 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:31:04.979934 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:31:04.979934 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:31:04.979934 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:31:04.979934 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:31:04.979934 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:31:04.979934 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:31:04.980958 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 17:31:04.980958 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 17:31:04.980958 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 17:31:04.980958 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 17:31:04.980958 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 17:31:04.980958 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 17:31:04.980958 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:31:04.980958 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Aug 24 17:31:04.980958 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 629/1258] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Aug 24 17:31:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 17:31:04.986374 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e9236f-4a43-432f-a4c7-32dd47321e0d None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:31:05.093914 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e9236f-4a43-432f-a4c7-32dd47321e0d None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:31:05.129586 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5519b1-f2fc-461f-9c64-d31bc06edd49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:31:05.130213 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5519b1-f2fc-461f-9c64-d31bc06edd49 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:05.130494 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5519b1-f2fc-461f-9c64-d31bc06edd49 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:05.130770 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5519b1-f2fc-461f-9c64-d31bc06edd49 None admin] PATH_INFO: `/v3/projects` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:05.131479 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5519b1-f2fc-461f-9c64-d31bc06edd49 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:31:05.131789 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5519b1-f2fc-461f-9c64-d31bc06edd49 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:31:05.133309 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5519b1-f2fc-461f-9c64-d31bc06edd49 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:31:05.147191 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 630/1259] 10.222.0.45 () {56 vars in 1148 bytes} [Wed Aug 24 17:31:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:31:05.153360 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6688f72-e336-4a9d-997e-8f3f356b024d None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:31:05.182534 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:31:05.265443 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6688f72-e336-4a9d-997e-8f3f356b024d None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:31:05.299643 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d72ada2-85a5-4c1c-90ee-4c9948383557 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:31:05.300264 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d72ada2-85a5-4c1c-90ee-4c9948383557 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:05.300540 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d72ada2-85a5-4c1c-90ee-4c9948383557 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:05.300842 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d72ada2-85a5-4c1c-90ee-4c9948383557 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:05.301585 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d72ada2-85a5-4c1c-90ee-4c9948383557 None admin] RBAC: Adding query filter params (role.id=e411ad9eb96e4e8d85b59b5b700ada7b, scope.project.id=2ad8a32bd9d2434097bd08b111914ca1, user.id=37def4721b554b9ebb7704a19642d15b) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:31:05.301893 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d72ada2-85a5-4c1c-90ee-4c9948383557 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:31:05.303343 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d72ada2-85a5-4c1c-90ee-4c9948383557 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:31:05.320583 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 630/1260] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Aug 24 17:31:05 2022] GET /identity/v3/role_assignments?user.id=37def4721b554b9ebb7704a19642d15b&scope.project.id=2ad8a32bd9d2434097bd08b111914ca1&role.id=e411ad9eb96e4e8d85b59b5b700ada7b => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:31:07.167127 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917051c3-ee96-4474-990a-27cdccebdf6e None None] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:07.167632 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917051c3-ee96-4474-990a-27cdccebdf6e None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:07.167741 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917051c3-ee96-4474-990a-27cdccebdf6e None None] PATH_INFO: `/` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:07.169057 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 631/1261] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:31:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 17:31:07.174389 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba311d0-25e9-4f22-aa98-f91dd42b56f2 None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:07.174809 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba311d0-25e9-4f22-aa98-f91dd42b56f2 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:07.174995 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba311d0-25e9-4f22-aa98-f91dd42b56f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:07.194487 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-5ba311d0-25e9-4f22-aa98-f91dd42b56f2 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:31:07.195734 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-5ba311d0-25e9-4f22-aa98-f91dd42b56f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:31:07.261119 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 631/1262] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:31:07 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 17:31:07.535470 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917051c3-ee96-4474-990a-27cdccebdf6e None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:31:07.546404 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:31:07.634460 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917051c3-ee96-4474-990a-27cdccebdf6e None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:31:07.666812 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04da1b24-f7ff-40c1-b632-3502f78514a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:31:07.667431 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04da1b24-f7ff-40c1-b632-3502f78514a9 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:07.667708 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04da1b24-f7ff-40c1-b632-3502f78514a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:07.667978 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04da1b24-f7ff-40c1-b632-3502f78514a9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:07.677758 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04da1b24-f7ff-40c1-b632-3502f78514a9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:31:07.682408 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application [None req-04da1b24-f7ff-40c1-b632-3502f78514a9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:31:07.682408 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:31:07.682408 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:31:07.682408 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:31:07.682408 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:31:07.682408 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:31:07.682408 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:31:07.682408 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:31:07.682408 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:31:07.682408 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:31:07.682408 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:31:07.682408 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:31:07.682408 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:31:07.682408 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:31:07.682408 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:31:07.682408 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:31:07.683480 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:31:07.683480 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:31:07.683480 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:31:07.683480 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:31:07.683480 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:31:07.683480 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:31:07.683480 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:31:07.683480 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:31:07.683480 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:31:07.683480 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:31:07.683480 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:31:07.683480 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application with Lock( Aug 24 17:31:07.683480 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:31:07.683480 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:31:07.683480 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:31:07.683480 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:31:07.684499 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:31:07.684499 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:31:07.684499 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:31:07.684499 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:31:07.684499 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:31:07.684499 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:31:07.684499 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:31:07.684499 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:31:07.684499 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:31:07.684499 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:31:07.684499 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:31:07.684499 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Aug 24 17:31:07.684499 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 632/1263] 10.222.0.45 () {56 vars in 1096 bytes} [Wed Aug 24 17:31:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:31:07.691830 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba311d0-25e9-4f22-aa98-f91dd42b56f2 None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:31:07.709905 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:31:07.786530 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba311d0-25e9-4f22-aa98-f91dd42b56f2 None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:31:07.818745 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38627c1b-39a3-46fa-9635-48cc51475e1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:31:07.819343 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38627c1b-39a3-46fa-9635-48cc51475e1f None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:07.819618 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38627c1b-39a3-46fa-9635-48cc51475e1f None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:07.819886 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38627c1b-39a3-46fa-9635-48cc51475e1f None admin] PATH_INFO: `/v3/roles` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:07.820536 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38627c1b-39a3-46fa-9635-48cc51475e1f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:31:07.820859 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38627c1b-39a3-46fa-9635-48cc51475e1f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:31:07.822207 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38627c1b-39a3-46fa-9635-48cc51475e1f None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:31:07.830405 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 632/1264] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Aug 24 17:31:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:31:07.836039 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04da1b24-f7ff-40c1-b632-3502f78514a9 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:31:07.900916 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:31:07.932644 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04da1b24-f7ff-40c1-b632-3502f78514a9 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:31:07.964807 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4591107-63c7-4c39-90ed-4183c608451e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:31:07.965413 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4591107-63c7-4c39-90ed-4183c608451e None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:07.965687 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4591107-63c7-4c39-90ed-4183c608451e None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:07.965977 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4591107-63c7-4c39-90ed-4183c608451e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:07.975400 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4591107-63c7-4c39-90ed-4183c608451e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:31:07.976851 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4591107-63c7-4c39-90ed-4183c608451e None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:31:07.991312 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application [None req-d4591107-63c7-4c39-90ed-4183c608451e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:31:07.991312 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:31:07.991312 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:31:07.991312 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:31:07.991312 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:31:07.991312 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:31:07.991312 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:31:07.991312 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:31:07.991312 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:31:07.991312 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:31:07.991312 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:31:07.991312 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:31:07.991312 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:31:07.991312 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:31:07.991312 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:31:07.991312 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:31:07.996649 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:31:07.996649 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:31:07.996649 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:31:07.996649 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:31:07.996649 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:31:07.996649 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:31:07.996649 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:31:07.996649 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application with Lock( Aug 24 17:31:07.996649 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:31:07.996649 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:31:07.996649 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:31:07.996649 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:31:07.996649 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:31:07.996649 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:31:07.996649 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:31:07.996649 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:31:07.996649 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:31:07.998224 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:31:07.998224 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:31:07.998224 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Aug 24 17:31:07.998224 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 633/1265] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 17:31:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:31:07.998629 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38627c1b-39a3-46fa-9635-48cc51475e1f None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:31:08.075582 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:31:08.094216 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38627c1b-39a3-46fa-9635-48cc51475e1f None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:31:08.126597 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d27dc64-ddac-469d-8aa7-b8201105d314 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:31:08.127181 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d27dc64-ddac-469d-8aa7-b8201105d314 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:08.127455 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d27dc64-ddac-469d-8aa7-b8201105d314 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:08.127722 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d27dc64-ddac-469d-8aa7-b8201105d314 None admin] PATH_INFO: `/v3/domains` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:08.128347 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d27dc64-ddac-469d-8aa7-b8201105d314 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:31:08.128670 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d27dc64-ddac-469d-8aa7-b8201105d314 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:31:08.130008 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d27dc64-ddac-469d-8aa7-b8201105d314 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:31:08.141134 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 633/1266] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 24 17:31:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:31:08.146865 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4591107-63c7-4c39-90ed-4183c608451e None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:31:08.253371 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4591107-63c7-4c39-90ed-4183c608451e None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:31:08.271621 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:31:08.286375 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354e62ad-ec08-466d-a628-c207d8a10cd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:31:08.287003 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354e62ad-ec08-466d-a628-c207d8a10cd1 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:08.287283 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354e62ad-ec08-466d-a628-c207d8a10cd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:08.287558 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354e62ad-ec08-466d-a628-c207d8a10cd1 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:08.295385 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354e62ad-ec08-466d-a628-c207d8a10cd1 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:31:08.296838 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354e62ad-ec08-466d-a628-c207d8a10cd1 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:31:08.307441 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application [None req-354e62ad-ec08-466d-a628-c207d8a10cd1 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 17:31:08.307441 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:31:08.307441 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:31:08.307441 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:31:08.307441 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:31:08.307441 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:31:08.307441 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:31:08.307441 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:31:08.307441 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:31:08.307441 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:31:08.307441 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:31:08.307441 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:31:08.307441 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 17:31:08.307441 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 17:31:08.307441 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 17:31:08.307441 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 17:31:08.308535 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:31:08.308535 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:31:08.308535 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 17:31:08.308535 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:31:08.308535 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 17:31:08.308535 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:31:08.308535 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:31:08.308535 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:31:08.308535 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:31:08.308535 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:31:08.308535 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:31:08.308535 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application with Lock( Aug 24 17:31:08.308535 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:31:08.308535 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:31:08.308535 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:31:08.308535 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:31:08.308535 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:31:08.308535 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:31:08.309505 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:31:08.309505 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:31:08.309505 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 17:31:08.309505 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 17:31:08.309505 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 17:31:08.309505 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 17:31:08.309505 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 17:31:08.309505 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 17:31:08.309505 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 17:31:08.309505 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Aug 24 17:31:08.309505 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 634/1267] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 17:31:08 2022] GET /identity/v3/users/placement => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:31:08.314454 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d27dc64-ddac-469d-8aa7-b8201105d314 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:31:08.408725 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d27dc64-ddac-469d-8aa7-b8201105d314 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:31:08.440650 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3323587c-993d-47e6-961a-490ea4dc9afa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:31:08.441236 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3323587c-993d-47e6-961a-490ea4dc9afa None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:08.441511 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3323587c-993d-47e6-961a-490ea4dc9afa None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:08.441778 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3323587c-993d-47e6-961a-490ea4dc9afa None admin] PATH_INFO: `/v3/users` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:08.442487 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3323587c-993d-47e6-961a-490ea4dc9afa None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:31:08.442789 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3323587c-993d-47e6-961a-490ea4dc9afa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:31:08.444192 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3323587c-993d-47e6-961a-490ea4dc9afa None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:31:08.452336 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:31:08.460934 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 634/1268] 10.222.0.45 () {56 vars in 1143 bytes} [Wed Aug 24 17:31:08 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:31:08.466967 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354e62ad-ec08-466d-a628-c207d8a10cd1 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:31:08.562283 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354e62ad-ec08-466d-a628-c207d8a10cd1 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:31:08.594494 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791b2583-9928-4aa2-8efe-48e650528ede None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:31:08.595101 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791b2583-9928-4aa2-8efe-48e650528ede None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:08.595375 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791b2583-9928-4aa2-8efe-48e650528ede None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:08.595642 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791b2583-9928-4aa2-8efe-48e650528ede None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:08.605642 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791b2583-9928-4aa2-8efe-48e650528ede None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:31:08.607717 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791b2583-9928-4aa2-8efe-48e650528ede None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:31:08.621337 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application [None req-791b2583-9928-4aa2-8efe-48e650528ede None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:31:08.621337 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:31:08.621337 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:31:08.621337 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:31:08.621337 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:31:08.621337 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:31:08.621337 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:31:08.621337 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:31:08.621337 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:31:08.621337 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:31:08.621337 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:31:08.621337 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:31:08.621337 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:31:08.621337 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:31:08.621337 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:31:08.621337 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:31:08.622409 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:31:08.622409 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:31:08.622409 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:31:08.622409 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:31:08.622409 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:31:08.622409 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:31:08.622409 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:31:08.622409 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application with Lock( Aug 24 17:31:08.622409 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:31:08.622409 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:31:08.622409 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:31:08.622409 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:31:08.622409 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:31:08.622409 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:31:08.622409 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:31:08.622409 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:31:08.622409 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:31:08.623306 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:31:08.623306 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:31:08.623306 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Aug 24 17:31:08.623306 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 635/1269] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 17:31:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:31:08.628130 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3323587c-993d-47e6-961a-490ea4dc9afa None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:31:08.723910 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3323587c-993d-47e6-961a-490ea4dc9afa None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:31:08.755902 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c09bf8b-fd7a-41a3-8d22-de46d0a06798 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:31:08.756495 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c09bf8b-fd7a-41a3-8d22-de46d0a06798 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:08.756805 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c09bf8b-fd7a-41a3-8d22-de46d0a06798 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:08.757079 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c09bf8b-fd7a-41a3-8d22-de46d0a06798 None admin] PATH_INFO: `/v3/domains` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:08.757712 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c09bf8b-fd7a-41a3-8d22-de46d0a06798 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:31:08.758014 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c09bf8b-fd7a-41a3-8d22-de46d0a06798 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:31:08.759360 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c09bf8b-fd7a-41a3-8d22-de46d0a06798 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:31:08.770635 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 635/1270] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 24 17:31:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:31:08.776192 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791b2583-9928-4aa2-8efe-48e650528ede None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:31:08.806774 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:31:08.872248 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791b2583-9928-4aa2-8efe-48e650528ede None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:31:08.904529 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7799ef-b5c5-42f7-8eeb-432b14a222b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:31:08.905171 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7799ef-b5c5-42f7-8eeb-432b14a222b4 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:08.905443 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7799ef-b5c5-42f7-8eeb-432b14a222b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:08.905712 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7799ef-b5c5-42f7-8eeb-432b14a222b4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:08.912010 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7799ef-b5c5-42f7-8eeb-432b14a222b4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:31:08.913471 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7799ef-b5c5-42f7-8eeb-432b14a222b4 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:31:08.924082 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application [None req-af7799ef-b5c5-42f7-8eeb-432b14a222b4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:31:08.924082 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:31:08.924082 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:31:08.924082 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:31:08.924082 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:31:08.924082 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:31:08.924082 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:31:08.924082 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:31:08.924082 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:31:08.924082 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:31:08.924082 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:31:08.924082 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:31:08.924082 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 17:31:08.924082 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 17:31:08.924082 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 17:31:08.925099 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 17:31:08.925099 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:31:08.925099 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:31:08.925099 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:31:08.925099 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:31:08.925099 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:31:08.925099 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:31:08.925099 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:31:08.925099 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application with Lock( Aug 24 17:31:08.925099 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:31:08.925099 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:31:08.925099 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:31:08.925099 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:31:08.925099 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:31:08.925099 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:31:08.925099 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:31:08.925099 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:31:08.926030 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 17:31:08.926030 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 17:31:08.926030 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 17:31:08.926030 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 17:31:08.926030 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 17:31:08.926030 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 17:31:08.926030 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:31:08.926030 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Aug 24 17:31:08.926030 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 636/1271] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Aug 24 17:31:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:31:08.931015 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c09bf8b-fd7a-41a3-8d22-de46d0a06798 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:31:08.987750 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:31:09.029476 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c09bf8b-fd7a-41a3-8d22-de46d0a06798 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:31:09.061998 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455e7244-6cdb-43bb-99c2-fdeb2187b3a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:31:09.062583 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455e7244-6cdb-43bb-99c2-fdeb2187b3a4 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:09.062860 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455e7244-6cdb-43bb-99c2-fdeb2187b3a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:09.063129 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455e7244-6cdb-43bb-99c2-fdeb2187b3a4 None admin] PATH_INFO: `/v3/projects` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:09.063811 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455e7244-6cdb-43bb-99c2-fdeb2187b3a4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:31:09.064119 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455e7244-6cdb-43bb-99c2-fdeb2187b3a4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:31:09.065560 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455e7244-6cdb-43bb-99c2-fdeb2187b3a4 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:31:09.077481 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 636/1272] 10.222.0.45 () {56 vars in 1148 bytes} [Wed Aug 24 17:31:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:31:09.083379 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7799ef-b5c5-42f7-8eeb-432b14a222b4 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:31:09.170260 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:31:09.179503 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7799ef-b5c5-42f7-8eeb-432b14a222b4 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:31:09.218009 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b119c058-eb53-4ba0-a23b-e765f14faf67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:31:09.218661 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b119c058-eb53-4ba0-a23b-e765f14faf67 None admin] REQUEST_METHOD: `PUT` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:09.218935 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b119c058-eb53-4ba0-a23b-e765f14faf67 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:09.219201 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b119c058-eb53-4ba0-a23b-e765f14faf67 None admin] PATH_INFO: `/v3/projects/2ad8a32bd9d2434097bd08b111914ca1/users/37def4721b554b9ebb7704a19642d15b/roles/e411ad9eb96e4e8d85b59b5b700ada7b` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:09.221083 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b119c058-eb53-4ba0-a23b-e765f14faf67 None admin] RBAC: Authorizing `identity:create_grant(project_id=2ad8a32bd9d2434097bd08b111914ca1, user_id=37def4721b554b9ebb7704a19642d15b, role_id=e411ad9eb96e4e8d85b59b5b700ada7b)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:31:09.222504 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b119c058-eb53-4ba0-a23b-e765f14faf67 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:31:09.234319 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 637/1273] 10.222.0.45 () {58 vars in 1439 bytes} [Wed Aug 24 17:31:09 2022] PUT /identity/v3/projects/2ad8a32bd9d2434097bd08b111914ca1/users/37def4721b554b9ebb7704a19642d15b/roles/e411ad9eb96e4e8d85b59b5b700ada7b => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:31:11.121436 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d956c3d1-8a14-4d7d-bb0e-040ceb3b8fb6 None None] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:11.121933 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d956c3d1-8a14-4d7d-bb0e-040ceb3b8fb6 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:11.121933 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d956c3d1-8a14-4d7d-bb0e-040ceb3b8fb6 None None] PATH_INFO: `/` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:11.123245 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 637/1274] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:31:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:31:11.128615 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882470cd-63ef-433e-8826-019c0556c3fa None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:11.129016 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882470cd-63ef-433e-8826-019c0556c3fa None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:11.129163 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882470cd-63ef-433e-8826-019c0556c3fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:11.147249 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-882470cd-63ef-433e-8826-019c0556c3fa None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:31:11.148428 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-882470cd-63ef-433e-8826-019c0556c3fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:31:11.222477 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 638/1275] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:31:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:31:11.502337 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d956c3d1-8a14-4d7d-bb0e-040ceb3b8fb6 None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:31:11.600517 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d956c3d1-8a14-4d7d-bb0e-040ceb3b8fb6 None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:31:11.618084 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:31:11.635033 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d1f850-b970-4115-aae3-9e38fe1223ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:31:11.635645 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d1f850-b970-4115-aae3-9e38fe1223ea None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:11.635923 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d1f850-b970-4115-aae3-9e38fe1223ea None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:11.636194 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d1f850-b970-4115-aae3-9e38fe1223ea None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:11.646458 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d1f850-b970-4115-aae3-9e38fe1223ea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:31:11.651057 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application [None req-16d1f850-b970-4115-aae3-9e38fe1223ea None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:31:11.651057 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:31:11.651057 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:31:11.651057 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:31:11.651057 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:31:11.651057 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:31:11.651057 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:31:11.651057 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:31:11.651057 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:31:11.651057 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:31:11.651057 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:31:11.651057 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:31:11.651057 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:31:11.651057 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:31:11.651057 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:31:11.651057 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:31:11.652132 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:31:11.652132 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:31:11.652132 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:31:11.652132 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:31:11.652132 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:31:11.652132 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:31:11.652132 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:31:11.652132 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:31:11.652132 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:31:11.652132 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:31:11.652132 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:31:11.652132 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application with Lock( Aug 24 17:31:11.652132 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:31:11.652132 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:31:11.652132 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:31:11.652132 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:31:11.653427 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:31:11.653427 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:31:11.653427 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:31:11.653427 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:31:11.653427 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:31:11.653427 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:31:11.653427 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:31:11.653427 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:31:11.653427 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:31:11.653427 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:31:11.653427 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:31:11.653427 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Aug 24 17:31:11.653427 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 638/1276] 10.222.0.45 () {56 vars in 1096 bytes} [Wed Aug 24 17:31:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:31:11.658052 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882470cd-63ef-433e-8826-019c0556c3fa None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:31:11.735867 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:31:11.754578 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882470cd-63ef-433e-8826-019c0556c3fa None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:31:11.786646 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038b2269-16c7-4fe0-af93-fc60553fb6d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:31:11.787243 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038b2269-16c7-4fe0-af93-fc60553fb6d9 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:11.787519 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038b2269-16c7-4fe0-af93-fc60553fb6d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:11.787789 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038b2269-16c7-4fe0-af93-fc60553fb6d9 None admin] PATH_INFO: `/v3/roles` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:11.788448 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038b2269-16c7-4fe0-af93-fc60553fb6d9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:31:11.788775 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038b2269-16c7-4fe0-af93-fc60553fb6d9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:31:11.790138 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038b2269-16c7-4fe0-af93-fc60553fb6d9 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:31:11.798433 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 639/1277] 10.222.0.45 () {56 vars in 1099 bytes} [Wed Aug 24 17:31:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:31:11.804063 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d1f850-b970-4115-aae3-9e38fe1223ea None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:31:11.898872 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d1f850-b970-4115-aae3-9e38fe1223ea None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:31:11.930838 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c389e6f-5f58-4d88-9dbe-0f41234ff209 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:31:11.931450 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c389e6f-5f58-4d88-9dbe-0f41234ff209 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:11.931733 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c389e6f-5f58-4d88-9dbe-0f41234ff209 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:11.932006 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c389e6f-5f58-4d88-9dbe-0f41234ff209 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:11.942526 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c389e6f-5f58-4d88-9dbe-0f41234ff209 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:31:11.943951 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c389e6f-5f58-4d88-9dbe-0f41234ff209 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:31:11.957385 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application [None req-3c389e6f-5f58-4d88-9dbe-0f41234ff209 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:31:11.957385 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:31:11.957385 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:31:11.957385 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:31:11.957385 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:31:11.957385 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:31:11.957385 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:31:11.957385 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:31:11.957385 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:31:11.957385 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:31:11.957385 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:31:11.957385 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:31:11.957385 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:31:11.957385 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:31:11.957385 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:31:11.957385 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:31:11.958477 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:31:11.958477 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:31:11.958477 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:31:11.958477 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:31:11.958477 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:31:11.958477 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:31:11.958477 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:31:11.958477 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application with Lock( Aug 24 17:31:11.958477 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:31:11.958477 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:31:11.958477 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:31:11.958477 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:31:11.958477 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:31:11.958477 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:31:11.958477 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:31:11.958477 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:31:11.958477 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:31:11.959360 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:31:11.959360 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:31:11.959360 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Aug 24 17:31:11.959360 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 639/1278] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 17:31:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:31:11.964242 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038b2269-16c7-4fe0-af93-fc60553fb6d9 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:31:12.059948 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038b2269-16c7-4fe0-af93-fc60553fb6d9 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:31:12.091415 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44fb339-2e45-4bca-b693-a95bc7958919 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:31:12.092009 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44fb339-2e45-4bca-b693-a95bc7958919 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:12.092283 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44fb339-2e45-4bca-b693-a95bc7958919 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:12.092576 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44fb339-2e45-4bca-b693-a95bc7958919 None admin] PATH_INFO: `/v3/domains` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:12.093205 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44fb339-2e45-4bca-b693-a95bc7958919 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:31:12.093501 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44fb339-2e45-4bca-b693-a95bc7958919 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:31:12.094848 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44fb339-2e45-4bca-b693-a95bc7958919 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:31:12.105916 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 640/1279] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 24 17:31:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:31:12.112142 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c389e6f-5f58-4d88-9dbe-0f41234ff209 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:31:12.141725 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:31:12.213504 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c389e6f-5f58-4d88-9dbe-0f41234ff209 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:31:12.246434 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f5d0ac-3b9a-4d9f-bec7-a24caccbbccb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:31:12.247055 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f5d0ac-3b9a-4d9f-bec7-a24caccbbccb None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:12.247336 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f5d0ac-3b9a-4d9f-bec7-a24caccbbccb None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:12.247610 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f5d0ac-3b9a-4d9f-bec7-a24caccbbccb None admin] PATH_INFO: `/v3/users/placement` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:12.254667 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f5d0ac-3b9a-4d9f-bec7-a24caccbbccb None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:31:12.256093 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f5d0ac-3b9a-4d9f-bec7-a24caccbbccb None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:31:12.266702 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application [None req-01f5d0ac-3b9a-4d9f-bec7-a24caccbbccb None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 17:31:12.266702 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:31:12.266702 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:31:12.266702 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:31:12.266702 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:31:12.266702 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:31:12.266702 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:31:12.266702 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:31:12.266702 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:31:12.266702 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:31:12.266702 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:31:12.266702 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:31:12.266702 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 17:31:12.266702 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 17:31:12.266702 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 17:31:12.266702 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 17:31:12.267784 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:31:12.267784 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:31:12.267784 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 17:31:12.267784 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:31:12.267784 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 17:31:12.267784 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:31:12.267784 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:31:12.267784 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:31:12.267784 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:31:12.267784 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:31:12.267784 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:31:12.267784 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application with Lock( Aug 24 17:31:12.267784 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:31:12.267784 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:31:12.267784 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:31:12.267784 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:31:12.267784 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:31:12.267784 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:31:12.268756 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:31:12.268756 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:31:12.268756 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 17:31:12.268756 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 17:31:12.268756 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 17:31:12.268756 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 17:31:12.268756 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 17:31:12.268756 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 17:31:12.268756 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 17:31:12.268756 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Aug 24 17:31:12.268756 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 640/1280] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 17:31:12 2022] GET /identity/v3/users/placement => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:31:12.273728 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44fb339-2e45-4bca-b693-a95bc7958919 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:31:12.277480 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:31:12.370589 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44fb339-2e45-4bca-b693-a95bc7958919 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:31:12.402796 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1958c508-e224-4b90-a44f-5412447b7390 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:31:12.403388 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1958c508-e224-4b90-a44f-5412447b7390 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:12.404210 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1958c508-e224-4b90-a44f-5412447b7390 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:12.404483 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1958c508-e224-4b90-a44f-5412447b7390 None admin] PATH_INFO: `/v3/users` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:12.405229 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1958c508-e224-4b90-a44f-5412447b7390 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:31:12.405534 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1958c508-e224-4b90-a44f-5412447b7390 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:31:12.406949 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1958c508-e224-4b90-a44f-5412447b7390 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:31:12.423565 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 641/1281] 10.222.0.45 () {56 vars in 1143 bytes} [Wed Aug 24 17:31:12 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:31:12.429349 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f5d0ac-3b9a-4d9f-bec7-a24caccbbccb None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:31:12.515831 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:31:12.525306 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f5d0ac-3b9a-4d9f-bec7-a24caccbbccb None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:31:12.557069 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159d0a43-f515-49a8-a06d-980413e4ca1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:31:12.557655 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159d0a43-f515-49a8-a06d-980413e4ca1b None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:12.557932 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159d0a43-f515-49a8-a06d-980413e4ca1b None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:12.558199 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159d0a43-f515-49a8-a06d-980413e4ca1b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:12.622825 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159d0a43-f515-49a8-a06d-980413e4ca1b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:31:12.624202 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159d0a43-f515-49a8-a06d-980413e4ca1b None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:31:12.706421 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:31:12.716450 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 641/1282] 10.222.0.45 () {58 vars in 1282 bytes} [Wed Aug 24 17:31:12 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 17:31:12.722224 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1958c508-e224-4b90-a44f-5412447b7390 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:31:12.782609 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:31:12.820502 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1958c508-e224-4b90-a44f-5412447b7390 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:31:12.853471 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58857df6-0ac5-4c4a-af0e-d1bae51e6830 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:31:12.854092 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58857df6-0ac5-4c4a-af0e-d1bae51e6830 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:12.854370 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58857df6-0ac5-4c4a-af0e-d1bae51e6830 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:12.854641 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58857df6-0ac5-4c4a-af0e-d1bae51e6830 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:12.865112 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58857df6-0ac5-4c4a-af0e-d1bae51e6830 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:31:12.866542 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58857df6-0ac5-4c4a-af0e-d1bae51e6830 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:31:12.880542 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application [None req-58857df6-0ac5-4c4a-af0e-d1bae51e6830 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:31:12.880542 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:31:12.880542 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:31:12.880542 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:31:12.880542 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:31:12.880542 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:31:12.880542 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:31:12.880542 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:31:12.880542 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:31:12.880542 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:31:12.880542 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:31:12.880542 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:31:12.880542 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:31:12.880542 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:31:12.880542 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:31:12.880542 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:31:12.881642 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:31:12.881642 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:31:12.881642 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:31:12.881642 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:31:12.881642 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:31:12.881642 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:31:12.881642 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:31:12.881642 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application with Lock( Aug 24 17:31:12.881642 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:31:12.881642 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:31:12.881642 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:31:12.881642 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:31:12.881642 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:31:12.881642 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:31:12.881642 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:31:12.881642 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:31:12.881642 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:31:12.882799 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:31:12.882799 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:31:12.882799 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Aug 24 17:31:12.882799 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 642/1283] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 17:31:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:31:12.887212 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159d0a43-f515-49a8-a06d-980413e4ca1b None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:31:12.981141 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159d0a43-f515-49a8-a06d-980413e4ca1b None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:31:13.013658 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1794c40b-39d5-4a52-8fe9-2f90a7119178 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:31:13.014245 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1794c40b-39d5-4a52-8fe9-2f90a7119178 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:13.014522 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1794c40b-39d5-4a52-8fe9-2f90a7119178 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:13.014791 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1794c40b-39d5-4a52-8fe9-2f90a7119178 None admin] PATH_INFO: `/v3/domains` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:13.015423 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1794c40b-39d5-4a52-8fe9-2f90a7119178 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:31:13.015725 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1794c40b-39d5-4a52-8fe9-2f90a7119178 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:31:13.017109 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1794c40b-39d5-4a52-8fe9-2f90a7119178 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:31:13.027705 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 642/1284] 10.222.0.45 () {56 vars in 1109 bytes} [Wed Aug 24 17:31:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:31:13.033659 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58857df6-0ac5-4c4a-af0e-d1bae51e6830 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:31:13.129793 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58857df6-0ac5-4c4a-af0e-d1bae51e6830 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:31:13.143244 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:31:13.162647 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275f2512-4713-40c1-9eff-6b38d45137a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:31:13.163261 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275f2512-4713-40c1-9eff-6b38d45137a5 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:13.163538 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275f2512-4713-40c1-9eff-6b38d45137a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:13.163809 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275f2512-4713-40c1-9eff-6b38d45137a5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:13.170656 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275f2512-4713-40c1-9eff-6b38d45137a5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:31:13.172083 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275f2512-4713-40c1-9eff-6b38d45137a5 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:31:13.184579 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application [None req-275f2512-4713-40c1-9eff-6b38d45137a5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:31:13.184579 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:31:13.184579 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:31:13.184579 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:31:13.184579 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:31:13.184579 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:31:13.184579 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:31:13.184579 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:31:13.184579 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:31:13.184579 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:31:13.184579 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:31:13.184579 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:31:13.184579 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 17:31:13.184579 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 17:31:13.184579 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 17:31:13.186244 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 17:31:13.186244 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:31:13.186244 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:31:13.186244 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:31:13.186244 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:31:13.186244 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:31:13.186244 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:31:13.186244 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:31:13.186244 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application with Lock( Aug 24 17:31:13.186244 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:31:13.186244 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:31:13.186244 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:31:13.186244 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:31:13.186244 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:31:13.186244 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:31:13.186244 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:31:13.186244 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:31:13.187350 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 17:31:13.187350 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 17:31:13.187350 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 17:31:13.187350 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 17:31:13.187350 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 17:31:13.187350 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 17:31:13.187350 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:31:13.187350 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: ERROR keystone.server.flask.application Aug 24 17:31:13.187350 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 643/1285] 10.222.0.45 () {56 vars in 1111 bytes} [Wed Aug 24 17:31:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:31:13.192915 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1794c40b-39d5-4a52-8fe9-2f90a7119178 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:31:13.242557 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:31:13.290517 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1794c40b-39d5-4a52-8fe9-2f90a7119178 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:31:13.323127 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9a7e4d-59ba-4c15-bc73-7ed4504e43d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:31:13.323723 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9a7e4d-59ba-4c15-bc73-7ed4504e43d7 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:13.324006 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9a7e4d-59ba-4c15-bc73-7ed4504e43d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:13.324277 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9a7e4d-59ba-4c15-bc73-7ed4504e43d7 None admin] PATH_INFO: `/v3/projects` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:13.324987 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9a7e4d-59ba-4c15-bc73-7ed4504e43d7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:31:13.325294 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9a7e4d-59ba-4c15-bc73-7ed4504e43d7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:31:13.326730 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9a7e4d-59ba-4c15-bc73-7ed4504e43d7 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:31:13.338121 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 643/1286] 10.222.0.45 () {56 vars in 1148 bytes} [Wed Aug 24 17:31:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:31:13.344229 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275f2512-4713-40c1-9eff-6b38d45137a5 None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:31:13.439989 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275f2512-4713-40c1-9eff-6b38d45137a5 None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:31:13.471059 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a3abc2-20e1-4b04-82a4-68add839fa11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:31:13.471639 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a3abc2-20e1-4b04-82a4-68add839fa11 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:13.471912 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a3abc2-20e1-4b04-82a4-68add839fa11 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:13.472182 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a3abc2-20e1-4b04-82a4-68add839fa11 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:13.472917 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a3abc2-20e1-4b04-82a4-68add839fa11 None admin] RBAC: Adding query filter params (role.id=e411ad9eb96e4e8d85b59b5b700ada7b, scope.project.id=2ad8a32bd9d2434097bd08b111914ca1, user.id=37def4721b554b9ebb7704a19642d15b) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:31:13.473226 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a3abc2-20e1-4b04-82a4-68add839fa11 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:31:13.474634 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a3abc2-20e1-4b04-82a4-68add839fa11 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:31:13.480072 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 644/1287] 10.222.0.45 () {56 vars in 1374 bytes} [Wed Aug 24 17:31:13 2022] GET /identity/v3/role_assignments?user.id=37def4721b554b9ebb7704a19642d15b&scope.project.id=2ad8a32bd9d2434097bd08b111914ca1&role.id=e411ad9eb96e4e8d85b59b5b700ada7b => generated 608 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:31:15.379560 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98498bb2-1504-4af9-9430-b4aaa792ed0d None None] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:15.380151 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98498bb2-1504-4af9-9430-b4aaa792ed0d None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:15.380151 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98498bb2-1504-4af9-9430-b4aaa792ed0d None None] PATH_INFO: `/` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:15.381480 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 644/1288] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:31:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:31:15.386643 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbd4860-6128-4a16-8348-818616ece1d7 None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:15.387081 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbd4860-6128-4a16-8348-818616ece1d7 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:15.387181 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbd4860-6128-4a16-8348-818616ece1d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:15.405748 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-7cbd4860-6128-4a16-8348-818616ece1d7 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:31:15.406923 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-7cbd4860-6128-4a16-8348-818616ece1d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:31:15.434927 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:31:15.471589 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 645/1289] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:31:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:31:15.736264 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98498bb2-1504-4af9-9430-b4aaa792ed0d None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:31:15.834747 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98498bb2-1504-4af9-9430-b4aaa792ed0d None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:31:15.844830 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:31:15.867694 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8effc86-8d8c-4b58-8dbc-d364136394b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:31:15.868311 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8effc86-8d8c-4b58-8dbc-d364136394b3 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:15.868615 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8effc86-8d8c-4b58-8dbc-d364136394b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:15.868887 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8effc86-8d8c-4b58-8dbc-d364136394b3 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:15.869469 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8effc86-8d8c-4b58-8dbc-d364136394b3 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:31:15.870818 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8effc86-8d8c-4b58-8dbc-d364136394b3 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:31:15.879660 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application [None req-a8effc86-8d8c-4b58-8dbc-d364136394b3 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 17:31:15.879660 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:31:15.879660 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 17:31:15.879660 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 17:31:15.879660 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 17:31:15.879660 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 17:31:15.879660 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 17:31:15.879660 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:31:15.879660 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 17:31:15.879660 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Aug 24 17:31:15.879660 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 17:31:15.879660 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Aug 24 17:31:15.879660 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:31:15.879660 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:31:15.879660 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:31:15.879660 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:31:15.879660 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:31:15.880906 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:31:15.880906 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:31:15.880906 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:31:15.880906 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:31:15.880906 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:31:15.880906 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:31:15.880906 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 17:31:15.880906 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 17:31:15.880906 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 17:31:15.880906 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 17:31:15.880906 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:31:15.880906 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:31:15.880906 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:31:15.880906 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:31:15.880906 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:31:15.880906 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:31:15.881963 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:31:15.881963 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application with Lock( Aug 24 17:31:15.881963 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:31:15.881963 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:31:15.881963 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:31:15.881963 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:31:15.881963 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:31:15.881963 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:31:15.881963 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:31:15.881963 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:31:15.881963 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 17:31:15.881963 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:31:15.881963 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 17:31:15.881963 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Aug 24 17:31:15.881963 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 645/1290] 10.222.0.45 () {56 vars in 1117 bytes} [Wed Aug 24 17:31:15 2022] GET /identity/v3/services/placement => generated 90 bytes in 145 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 17:31:15.886632 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbd4860-6128-4a16-8348-818616ece1d7 None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:31:15.980775 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbd4860-6128-4a16-8348-818616ece1d7 None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:31:16.013972 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c385c5b4-f2a7-40bb-be41-31ee905cc3ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:31:16.014509 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c385c5b4-f2a7-40bb-be41-31ee905cc3ca None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:16.014788 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c385c5b4-f2a7-40bb-be41-31ee905cc3ca None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:16.015060 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c385c5b4-f2a7-40bb-be41-31ee905cc3ca None admin] PATH_INFO: `/v3/services` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:16.015681 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c385c5b4-f2a7-40bb-be41-31ee905cc3ca None admin] RBAC: Adding query filter params (name=placement) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:31:16.015983 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c385c5b4-f2a7-40bb-be41-31ee905cc3ca None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:31:16.017377 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c385c5b4-f2a7-40bb-be41-31ee905cc3ca None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:31:16.021983 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 646/1291] 10.222.0.45 () {56 vars in 1116 bytes} [Wed Aug 24 17:31:15 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:31:16.027649 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8effc86-8d8c-4b58-8dbc-d364136394b3 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:31:16.124237 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8effc86-8d8c-4b58-8dbc-d364136394b3 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:31:16.156591 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dab6f6-b2e9-4607-a50f-025fdb256ae7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:31:16.157181 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dab6f6-b2e9-4607-a50f-025fdb256ae7 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:16.157465 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dab6f6-b2e9-4607-a50f-025fdb256ae7 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:16.157738 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dab6f6-b2e9-4607-a50f-025fdb256ae7 None admin] PATH_INFO: `/v3/services` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:16.158359 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74dab6f6-b2e9-4607-a50f-025fdb256ae7 None admin] RBAC: Adding query filter params (type=placement) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:31:16.158661 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74dab6f6-b2e9-4607-a50f-025fdb256ae7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:31:16.160011 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74dab6f6-b2e9-4607-a50f-025fdb256ae7 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:31:16.164260 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 646/1292] 10.222.0.45 () {56 vars in 1116 bytes} [Wed Aug 24 17:31:16 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:31:18.061978 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b624345b-9ee7-4642-9452-8916a8f68dea None None] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:18.062477 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b624345b-9ee7-4642-9452-8916a8f68dea None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:18.062477 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b624345b-9ee7-4642-9452-8916a8f68dea None None] PATH_INFO: `/` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:18.063777 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 647/1293] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:31:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:31:18.069042 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73729fb8-049e-4cb8-a30a-e7ba04ccf15e None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:18.069435 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73729fb8-049e-4cb8-a30a-e7ba04ccf15e None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:18.069601 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73729fb8-049e-4cb8-a30a-e7ba04ccf15e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:18.087670 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-73729fb8-049e-4cb8-a30a-e7ba04ccf15e None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:31:18.088885 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-73729fb8-049e-4cb8-a30a-e7ba04ccf15e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:31:18.117420 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:31:18.154423 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 647/1294] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:31:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:31:18.436627 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b624345b-9ee7-4642-9452-8916a8f68dea None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:31:18.466819 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:31:18.535281 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b624345b-9ee7-4642-9452-8916a8f68dea None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:31:18.568635 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2be26e-706d-4bee-b705-49d145bfe10e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:31:18.569243 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2be26e-706d-4bee-b705-49d145bfe10e None admin] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:18.569522 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2be26e-706d-4bee-b705-49d145bfe10e None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:18.569797 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2be26e-706d-4bee-b705-49d145bfe10e None admin] PATH_INFO: `/v3/services` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:18.570451 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2be26e-706d-4bee-b705-49d145bfe10e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:31:18.571818 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2be26e-706d-4bee-b705-49d145bfe10e None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:31:18.585616 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 648/1295] 10.222.0.45 () {60 vars in 1141 bytes} [Wed Aug 24 17:31:18 2022] POST /identity/v3/services => generated 246 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:31:20.476514 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33df4041-d89c-4c63-b403-1e0a4e48347a None None] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:20.477091 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33df4041-d89c-4c63-b403-1e0a4e48347a None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:20.477091 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33df4041-d89c-4c63-b403-1e0a4e48347a None None] PATH_INFO: `/` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:20.478415 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 648/1296] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:31:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:31:20.484245 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f95945-82c8-45fb-8674-e104bc44b59b None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:20.484643 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f95945-82c8-45fb-8674-e104bc44b59b None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:20.484841 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f95945-82c8-45fb-8674-e104bc44b59b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:20.502548 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-83f95945-82c8-45fb-8674-e104bc44b59b None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:31:20.503734 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-83f95945-82c8-45fb-8674-e104bc44b59b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:31:20.573299 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 649/1297] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:31:20 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:31:20.833741 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33df4041-d89c-4c63-b403-1e0a4e48347a None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:31:20.929524 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33df4041-d89c-4c63-b403-1e0a4e48347a None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:31:20.962119 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b87c357-0c37-4be0-bdd2-9c2a6b239973 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:31:20.962725 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b87c357-0c37-4be0-bdd2-9c2a6b239973 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:20.962998 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b87c357-0c37-4be0-bdd2-9c2a6b239973 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:20.963263 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b87c357-0c37-4be0-bdd2-9c2a6b239973 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:20.963837 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b87c357-0c37-4be0-bdd2-9c2a6b239973 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:31:20.965214 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b87c357-0c37-4be0-bdd2-9c2a6b239973 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:31:20.974000 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application [None req-9b87c357-0c37-4be0-bdd2-9c2a6b239973 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 17:31:20.974000 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:31:20.974000 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 17:31:20.974000 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 17:31:20.974000 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 17:31:20.974000 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 17:31:20.974000 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 17:31:20.974000 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:31:20.974000 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 17:31:20.974000 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Aug 24 17:31:20.974000 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 17:31:20.974000 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Aug 24 17:31:20.974000 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:31:20.974000 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:31:20.974000 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:31:20.974000 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:31:20.974000 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:31:20.975073 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:31:20.975073 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:31:20.975073 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:31:20.975073 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:31:20.975073 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:31:20.975073 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:31:20.975073 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 17:31:20.975073 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 17:31:20.975073 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 17:31:20.975073 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 17:31:20.975073 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:31:20.975073 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:31:20.975073 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:31:20.975073 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:31:20.975073 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:31:20.975073 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:31:20.975914 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:31:20.975914 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application with Lock( Aug 24 17:31:20.975914 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:31:20.975914 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:31:20.975914 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:31:20.975914 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:31:20.975914 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:31:20.975914 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:31:20.975914 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:31:20.975914 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:31:20.975914 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 17:31:20.975914 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:31:20.975914 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 17:31:20.975914 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Aug 24 17:31:20.975914 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 649/1298] 10.222.0.45 () {56 vars in 1117 bytes} [Wed Aug 24 17:31:20 2022] GET /identity/v3/services/placement => generated 90 bytes in 142 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:31:20.980922 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f95945-82c8-45fb-8674-e104bc44b59b None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:31:20.999357 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:31:21.078378 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f95945-82c8-45fb-8674-e104bc44b59b None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:31:21.110211 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22387309-20e8-421a-88cd-4da6ee7b163a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:31:21.110807 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22387309-20e8-421a-88cd-4da6ee7b163a None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:21.111132 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22387309-20e8-421a-88cd-4da6ee7b163a None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:21.111353 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22387309-20e8-421a-88cd-4da6ee7b163a None admin] PATH_INFO: `/v3/services` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:21.111973 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22387309-20e8-421a-88cd-4da6ee7b163a None admin] RBAC: Adding query filter params (name=placement) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:31:21.112273 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22387309-20e8-421a-88cd-4da6ee7b163a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:31:21.113920 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22387309-20e8-421a-88cd-4da6ee7b163a None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:31:21.118843 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 650/1299] 10.222.0.45 () {56 vars in 1116 bytes} [Wed Aug 24 17:31:20 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:31:21.125338 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b87c357-0c37-4be0-bdd2-9c2a6b239973 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:31:21.155785 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:31:21.239405 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b87c357-0c37-4be0-bdd2-9c2a6b239973 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:31:21.274489 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef37f9d-014a-4691-9379-804bddf49958 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:31:21.275081 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef37f9d-014a-4691-9379-804bddf49958 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:21.275356 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef37f9d-014a-4691-9379-804bddf49958 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:21.275624 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef37f9d-014a-4691-9379-804bddf49958 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:21.276314 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef37f9d-014a-4691-9379-804bddf49958 None admin] RBAC: Adding query filter params (interface=public, service_id=9132d177d4124e959324722e9c9c2ad6, region_id=RegionOne) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:31:21.276646 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef37f9d-014a-4691-9379-804bddf49958 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:31:21.277998 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef37f9d-014a-4691-9379-804bddf49958 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:31:21.282758 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 650/1300] 10.222.0.45 () {56 vars in 1251 bytes} [Wed Aug 24 17:31:21 2022] GET /identity/v3/endpoints?service_id=9132d177d4124e959324722e9c9c2ad6&interface=public®ion_id=RegionOne => generated 194 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:31:21.288428 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22387309-20e8-421a-88cd-4da6ee7b163a None admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:31:21.366634 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:31:21.384670 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22387309-20e8-421a-88cd-4da6ee7b163a None admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:31:21.417211 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29353db-9b6f-413a-84b0-ff8e9cba3e90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:31:21.417810 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29353db-9b6f-413a-84b0-ff8e9cba3e90 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:21.418087 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29353db-9b6f-413a-84b0-ff8e9cba3e90 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:21.418358 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29353db-9b6f-413a-84b0-ff8e9cba3e90 None admin] PATH_INFO: `/v3/services` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:21.418956 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29353db-9b6f-413a-84b0-ff8e9cba3e90 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:31:21.420918 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29353db-9b6f-413a-84b0-ff8e9cba3e90 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:31:21.427406 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 651/1301] 10.222.0.45 () {56 vars in 1087 bytes} [Wed Aug 24 17:31:21 2022] GET /identity/v3/services => generated 2191 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:31:23.279567 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4bdd4b-eaff-47d2-b310-0a4d49827a13 None None] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:23.280120 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4bdd4b-eaff-47d2-b310-0a4d49827a13 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:23.280120 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4bdd4b-eaff-47d2-b310-0a4d49827a13 None None] PATH_INFO: `/` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:23.281455 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 651/1302] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:31:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:31:23.286602 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7876ab9e-fa78-490a-a0fd-e666fb922126 None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:23.287020 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7876ab9e-fa78-490a-a0fd-e666fb922126 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:23.287154 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7876ab9e-fa78-490a-a0fd-e666fb922126 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:23.305312 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-7876ab9e-fa78-490a-a0fd-e666fb922126 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:31:23.306486 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-7876ab9e-fa78-490a-a0fd-e666fb922126 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:31:23.369937 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 652/1303] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:31:23 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:31:23.639925 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4bdd4b-eaff-47d2-b310-0a4d49827a13 None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:31:23.732569 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:31:23.739844 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4bdd4b-eaff-47d2-b310-0a4d49827a13 None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:31:23.772475 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b0d50b-3fde-4692-a9d0-a1a61aa33023 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:31:23.773128 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b0d50b-3fde-4692-a9d0-a1a61aa33023 None admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:23.773408 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b0d50b-3fde-4692-a9d0-a1a61aa33023 None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:23.773680 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b0d50b-3fde-4692-a9d0-a1a61aa33023 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:23.774263 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b0d50b-3fde-4692-a9d0-a1a61aa33023 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:31:23.775619 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b0d50b-3fde-4692-a9d0-a1a61aa33023 None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:31:23.784853 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application [None req-65b0d50b-3fde-4692-a9d0-a1a61aa33023 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 17:31:23.784853 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:31:23.784853 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 17:31:23.784853 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 17:31:23.784853 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 17:31:23.784853 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 17:31:23.784853 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 17:31:23.784853 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:31:23.784853 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 17:31:23.784853 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Aug 24 17:31:23.784853 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 17:31:23.784853 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Aug 24 17:31:23.784853 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:31:23.784853 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:31:23.784853 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:31:23.784853 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:31:23.784853 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:31:23.786003 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:31:23.786003 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:31:23.786003 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:31:23.786003 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:31:23.786003 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:31:23.786003 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:31:23.786003 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 17:31:23.786003 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 17:31:23.786003 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 17:31:23.786003 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 17:31:23.786003 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:31:23.786003 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:31:23.786003 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:31:23.786003 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:31:23.786003 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:31:23.786003 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:31:23.787170 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:31:23.787170 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application with Lock( Aug 24 17:31:23.787170 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:31:23.787170 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:31:23.787170 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:31:23.787170 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:31:23.787170 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:31:23.787170 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:31:23.787170 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:31:23.787170 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:31:23.787170 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 17:31:23.787170 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:31:23.787170 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 17:31:23.787170 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: ERROR keystone.server.flask.application Aug 24 17:31:23.787170 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 652/1304] 10.222.0.45 () {56 vars in 1117 bytes} [Wed Aug 24 17:31:23 2022] GET /identity/v3/services/placement => generated 90 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:31:23.791886 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7876ab9e-fa78-490a-a0fd-e666fb922126 None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:31:23.865611 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:31:23.888346 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7876ab9e-fa78-490a-a0fd-e666fb922126 None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:31:23.920920 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6489aa-7af2-42ed-afce-b54e9e78af21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:31:23.921518 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6489aa-7af2-42ed-afce-b54e9e78af21 None admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:23.921797 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6489aa-7af2-42ed-afce-b54e9e78af21 None admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:23.922067 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6489aa-7af2-42ed-afce-b54e9e78af21 None admin] PATH_INFO: `/v3/services` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:23.922691 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6489aa-7af2-42ed-afce-b54e9e78af21 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:31:23.922986 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6489aa-7af2-42ed-afce-b54e9e78af21 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:31:23.924348 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6489aa-7af2-42ed-afce-b54e9e78af21 None admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:31:23.929306 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 653/1305] 10.222.0.45 () {56 vars in 1116 bytes} [Wed Aug 24 17:31:23 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:31:23.935018 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b0d50b-3fde-4692-a9d0-a1a61aa33023 None admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:31:24.031647 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b0d50b-3fde-4692-a9d0-a1a61aa33023 None admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:31:24.063770 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6202965-a588-46f3-bd0d-50cbf0682deb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:31:24.064351 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6202965-a588-46f3-bd0d-50cbf0682deb None admin] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:24.064657 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6202965-a588-46f3-bd0d-50cbf0682deb None admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:24.064932 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6202965-a588-46f3-bd0d-50cbf0682deb None admin] PATH_INFO: `/v3/endpoints` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:24.065589 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6202965-a588-46f3-bd0d-50cbf0682deb None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:31:24.066943 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6202965-a588-46f3-bd0d-50cbf0682deb None admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:31:24.085766 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 653/1306] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 17:31:23 2022] POST /identity/v3/endpoints => generated 331 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:31:33.926369 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=30 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:31:34.081705 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:32:27.075505 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79de2984-bdee-4103-8179-8228958396e4 None None] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:32:27.076974 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79de2984-bdee-4103-8179-8228958396e4 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:32:27.077401 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79de2984-bdee-4103-8179-8228958396e4 None None] PATH_INFO: `/` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:32:27.082814 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 654/1307] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:32:27 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:32:27.084114 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0196785a-a13f-4c14-8462-63f4f065e7a0 None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:32:27.084537 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0196785a-a13f-4c14-8462-63f4f065e7a0 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:32:27.084669 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0196785a-a13f-4c14-8462-63f4f065e7a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:32:27.187612 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-0196785a-a13f-4c14-8462-63f4f065e7a0 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:32:27.188998 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-0196785a-a13f-4c14-8462-63f4f065e7a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:32:27.345450 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 654/1308] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:32:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:32:27.630347 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e6fe14-8fd6-40e9-9f8a-2c96db4be039 None None] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:32:27.630807 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e6fe14-8fd6-40e9-9f8a-2c96db4be039 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:32:27.631180 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e6fe14-8fd6-40e9-9f8a-2c96db4be039 None None] PATH_INFO: `/` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:32:27.632662 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 655/1309] 10.222.0.45 () {54 vars in 965 bytes} [Wed Aug 24 17:32:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:32:27.638695 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b31ca8-5a9b-4284-b91f-db73225fb15e None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:32:27.639058 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b31ca8-5a9b-4284-b91f-db73225fb15e None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:32:27.639409 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b31ca8-5a9b-4284-b91f-db73225fb15e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:32:27.715508 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-17b31ca8-5a9b-4284-b91f-db73225fb15e None None] MFA Rules not processed for user `43ad98a250a144099ccd18378aedd4b5`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:32:27.716779 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-17b31ca8-5a9b-4284-b91f-db73225fb15e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:32:27.737393 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:32:27.807597 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 655/1310] 10.222.0.45 () {58 vars in 1062 bytes} [Wed Aug 24 17:32:27 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:32:27.817139 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e6fe14-8fd6-40e9-9f8a-2c96db4be039 None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:32:27.819027 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-85e6fe14-8fd6-40e9-9f8a-2c96db4be039 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:32:27.940244 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e6fe14-8fd6-40e9-9f8a-2c96db4be039 None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:32:28.018815 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc140c8-3f92-4491-af30-3f599e37c51e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43ad98a250a144099ccd18378aedd4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:32:28.019429 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc140c8-3f92-4491-af30-3f599e37c51e service neutron] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:32:28.019706 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc140c8-3f92-4491-af30-3f599e37c51e service neutron] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:32:28.019977 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc140c8-3f92-4491-af30-3f599e37c51e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:32:28.021618 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-5dc140c8-3f92-4491-af30-3f599e37c51e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:32:28.080806 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:32:28.139612 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc140c8-3f92-4491-af30-3f599e37c51e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:32:28.141215 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc140c8-3f92-4491-af30-3f599e37c51e service neutron] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:32:28.161016 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 656/1311] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:32:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:32:30.359260 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9029b23-3b2a-44e7-b6e6-2b7b898f78b0 None None] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:32:30.359787 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9029b23-3b2a-44e7-b6e6-2b7b898f78b0 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:32:30.359787 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9029b23-3b2a-44e7-b6e6-2b7b898f78b0 None None] PATH_INFO: `/` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:32:30.361314 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 656/1312] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:32:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:32:30.367065 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f74180c-3bde-4bfa-9220-41465f67766c None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:32:30.367585 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f74180c-3bde-4bfa-9220-41465f67766c None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:32:30.367663 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f74180c-3bde-4bfa-9220-41465f67766c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:32:30.453783 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-5f74180c-3bde-4bfa-9220-41465f67766c None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:32:30.454987 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-5f74180c-3bde-4bfa-9220-41465f67766c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:32:30.554456 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 657/1313] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:32:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:32:30.855167 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3758e9ac-080a-496d-936e-05ed51bb0e70 None None] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:32:30.855453 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3758e9ac-080a-496d-936e-05ed51bb0e70 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:32:30.855731 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3758e9ac-080a-496d-936e-05ed51bb0e70 None None] PATH_INFO: `/` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:32:30.857761 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 657/1314] 10.222.0.45 () {54 vars in 965 bytes} [Wed Aug 24 17:32:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:32:30.863592 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5328ce89-d09c-4738-a1b2-e1a8c82dcdf3 None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:32:30.863826 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5328ce89-d09c-4738-a1b2-e1a8c82dcdf3 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:32:30.864101 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5328ce89-d09c-4738-a1b2-e1a8c82dcdf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:32:30.875863 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:32:30.937707 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-5328ce89-d09c-4738-a1b2-e1a8c82dcdf3 None None] MFA Rules not processed for user `43ad98a250a144099ccd18378aedd4b5`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:32:30.938893 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-5328ce89-d09c-4738-a1b2-e1a8c82dcdf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:32:31.011676 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 658/1315] 10.222.0.45 () {58 vars in 1062 bytes} [Wed Aug 24 17:32:30 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:32:31.020631 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3758e9ac-080a-496d-936e-05ed51bb0e70 None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:32:31.022137 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-3758e9ac-080a-496d-936e-05ed51bb0e70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:32:31.122494 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3758e9ac-080a-496d-936e-05ed51bb0e70 None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:32:31.190362 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:32:31.193109 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557b7c5e-3773-42ce-862a-0f8e73d7c670 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43ad98a250a144099ccd18378aedd4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:32:31.194010 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557b7c5e-3773-42ce-862a-0f8e73d7c670 service neutron] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:32:31.194524 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557b7c5e-3773-42ce-862a-0f8e73d7c670 service neutron] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:32:31.195037 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557b7c5e-3773-42ce-862a-0f8e73d7c670 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:32:31.196677 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-557b7c5e-3773-42ce-862a-0f8e73d7c670 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:32:31.302029 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557b7c5e-3773-42ce-862a-0f8e73d7c670 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:32:31.303440 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557b7c5e-3773-42ce-862a-0f8e73d7c670 service neutron] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:32:31.320976 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 658/1316] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:32:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:32:33.709332 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dbb2e7-7c1d-4f81-8f71-e04453dc6689 None None] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:32:33.709911 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dbb2e7-7c1d-4f81-8f71-e04453dc6689 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:32:33.709911 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dbb2e7-7c1d-4f81-8f71-e04453dc6689 None None] PATH_INFO: `/` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:32:33.711269 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 659/1317] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:32:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:32:33.716481 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15242e7f-6252-43c6-b896-82861857f7ab None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:32:33.716886 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15242e7f-6252-43c6-b896-82861857f7ab None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:32:33.717064 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15242e7f-6252-43c6-b896-82861857f7ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:32:33.782077 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-15242e7f-6252-43c6-b896-82861857f7ab None None] MFA Rules not processed for user `1f7a270ae41e45eab15c82f41826ca94`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:32:33.783305 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-15242e7f-6252-43c6-b896-82861857f7ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:32:33.833156 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:32:33.877282 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 659/1318] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:32:33 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:32:34.160222 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dbb2e7-7c1d-4f81-8f71-e04453dc6689 None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:32:34.161822 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-89dbb2e7-7c1d-4f81-8f71-e04453dc6689 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:32:34.270269 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dbb2e7-7c1d-4f81-8f71-e04453dc6689 None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:32:34.273028 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:32:34.340147 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d9146d-3551-40f0-bf94-6aab905ed8f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43ad98a250a144099ccd18378aedd4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:32:34.340769 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d9146d-3551-40f0-bf94-6aab905ed8f7 service neutron] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:32:34.341047 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d9146d-3551-40f0-bf94-6aab905ed8f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:32:34.341906 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d9146d-3551-40f0-bf94-6aab905ed8f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:32:34.343477 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-35d9146d-3551-40f0-bf94-6aab905ed8f7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:32:34.446907 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d9146d-3551-40f0-bf94-6aab905ed8f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:32:34.448260 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d9146d-3551-40f0-bf94-6aab905ed8f7 service neutron] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:32:34.466022 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 660/1319] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:32:34 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:32:37.013961 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9621df4-8528-471c-8e1c-e25c1d3f7362 None None] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:32:37.014462 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9621df4-8528-471c-8e1c-e25c1d3f7362 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:32:37.014462 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9621df4-8528-471c-8e1c-e25c1d3f7362 None None] PATH_INFO: `/` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:32:37.015785 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 660/1320] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:32:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:32:37.020930 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3fae54-4ecb-4861-908a-caf7e2352e16 None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:32:37.021354 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3fae54-4ecb-4861-908a-caf7e2352e16 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:32:37.021493 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3fae54-4ecb-4861-908a-caf7e2352e16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:32:37.061194 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:32:37.085966 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-da3fae54-4ecb-4861-908a-caf7e2352e16 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:32:37.087188 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-da3fae54-4ecb-4861-908a-caf7e2352e16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:32:37.182074 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 661/1321] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:32:37 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:32:37.462857 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9621df4-8528-471c-8e1c-e25c1d3f7362 None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:32:37.464391 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-f9621df4-8528-471c-8e1c-e25c1d3f7362 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:32:37.565164 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9621df4-8528-471c-8e1c-e25c1d3f7362 None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:32:37.635574 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2de5c96-b5ab-429c-8013-467a24284f32 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43ad98a250a144099ccd18378aedd4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:32:37.636173 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2de5c96-b5ab-429c-8013-467a24284f32 service neutron] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:32:37.636455 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2de5c96-b5ab-429c-8013-467a24284f32 service neutron] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:32:37.636755 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2de5c96-b5ab-429c-8013-467a24284f32 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:32:37.638895 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-d2de5c96-b5ab-429c-8013-467a24284f32 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:32:37.646063 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:32:37.742894 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2de5c96-b5ab-429c-8013-467a24284f32 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:32:37.744242 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2de5c96-b5ab-429c-8013-467a24284f32 service neutron] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:32:37.763227 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 661/1322] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:32:37 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:32:40.326494 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a779c273-399a-42fd-894b-be158c4fc3fd None None] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:32:40.327093 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a779c273-399a-42fd-894b-be158c4fc3fd None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:32:40.327093 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a779c273-399a-42fd-894b-be158c4fc3fd None None] PATH_INFO: `/` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:32:40.328359 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 662/1323] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:32:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:32:40.333742 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb958379-80b8-48ad-8622-5b2759d212d2 None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:32:40.334155 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb958379-80b8-48ad-8622-5b2759d212d2 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:32:40.334408 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb958379-80b8-48ad-8622-5b2759d212d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:32:40.400130 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-fb958379-80b8-48ad-8622-5b2759d212d2 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:32:40.401355 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-fb958379-80b8-48ad-8622-5b2759d212d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:32:40.460044 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:32:40.501311 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 662/1324] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:32:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:32:40.784110 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a779c273-399a-42fd-894b-be158c4fc3fd None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:32:40.785722 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-a779c273-399a-42fd-894b-be158c4fc3fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:32:40.888999 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a779c273-399a-42fd-894b-be158c4fc3fd None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:32:40.900847 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:32:40.962135 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6964472c-f3d3-4463-b4cf-f0b223b2c768 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43ad98a250a144099ccd18378aedd4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:32:40.962639 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6964472c-f3d3-4463-b4cf-f0b223b2c768 service neutron] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:32:40.962917 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6964472c-f3d3-4463-b4cf-f0b223b2c768 service neutron] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:32:40.963187 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6964472c-f3d3-4463-b4cf-f0b223b2c768 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:32:40.964819 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-6964472c-f3d3-4463-b4cf-f0b223b2c768 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:32:41.073037 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6964472c-f3d3-4463-b4cf-f0b223b2c768 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:32:41.074499 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6964472c-f3d3-4463-b4cf-f0b223b2c768 service neutron] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:32:41.092003 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 663/1325] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:32:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:32:43.032414 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dfd728-295b-4eac-9a29-e5e36a832668 None None] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:32:43.032953 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dfd728-295b-4eac-9a29-e5e36a832668 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:32:43.032953 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dfd728-295b-4eac-9a29-e5e36a832668 None None] PATH_INFO: `/` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:32:43.034246 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 663/1326] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:32:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:32:43.039522 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0dc0e7-34f8-495d-9bad-787a15a3b38b None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:32:43.039912 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0dc0e7-34f8-495d-9bad-787a15a3b38b None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:32:43.040075 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0dc0e7-34f8-495d-9bad-787a15a3b38b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:32:43.087292 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:32:43.105715 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-2b0dc0e7-34f8-495d-9bad-787a15a3b38b None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:32:43.107153 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-2b0dc0e7-34f8-495d-9bad-787a15a3b38b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:32:43.205050 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 664/1327] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:32:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:32:43.488653 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5dfd728-295b-4eac-9a29-e5e36a832668 None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:32:43.490178 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-e5dfd728-295b-4eac-9a29-e5e36a832668 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:32:43.598540 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5dfd728-295b-4eac-9a29-e5e36a832668 None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:32:43.671647 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85a9fc3-0b78-4fb3-ab89-5b31422c1574 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43ad98a250a144099ccd18378aedd4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:32:43.672255 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85a9fc3-0b78-4fb3-ab89-5b31422c1574 service neutron] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:32:43.672537 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85a9fc3-0b78-4fb3-ab89-5b31422c1574 service neutron] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:32:43.672840 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85a9fc3-0b78-4fb3-ab89-5b31422c1574 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:32:43.674418 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-c85a9fc3-0b78-4fb3-ab89-5b31422c1574 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:32:43.690807 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:32:43.782038 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85a9fc3-0b78-4fb3-ab89-5b31422c1574 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:32:43.783476 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85a9fc3-0b78-4fb3-ab89-5b31422c1574 service neutron] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:32:43.802028 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 664/1328] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:32:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:32:45.751417 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca39140-636d-4213-a75d-508a955a3ea3 None None] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:32:45.751908 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca39140-636d-4213-a75d-508a955a3ea3 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:32:45.752031 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca39140-636d-4213-a75d-508a955a3ea3 None None] PATH_INFO: `/` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:32:45.753317 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 665/1329] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:32:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:32:45.758623 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e777f5a-7e8b-4358-86ed-7c9011d787ab None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:32:45.759053 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e777f5a-7e8b-4358-86ed-7c9011d787ab None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:32:45.759181 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e777f5a-7e8b-4358-86ed-7c9011d787ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:32:45.831908 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-9e777f5a-7e8b-4358-86ed-7c9011d787ab None None] MFA Rules not processed for user `1f7a270ae41e45eab15c82f41826ca94`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:32:45.833142 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-9e777f5a-7e8b-4358-86ed-7c9011d787ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:32:45.895130 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:32:45.929145 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 665/1330] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:32:45 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:32:46.211101 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca39140-636d-4213-a75d-508a955a3ea3 None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:32:46.212685 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-3ca39140-636d-4213-a75d-508a955a3ea3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:32:46.318742 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca39140-636d-4213-a75d-508a955a3ea3 None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:32:46.338563 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:32:46.399854 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e20c35-1459-49df-901e-e5bc19429a5c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43ad98a250a144099ccd18378aedd4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:32:46.400462 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e20c35-1459-49df-901e-e5bc19429a5c service neutron] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:32:46.400854 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e20c35-1459-49df-901e-e5bc19429a5c service neutron] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:32:46.401125 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e20c35-1459-49df-901e-e5bc19429a5c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:32:46.402714 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-92e20c35-1459-49df-901e-e5bc19429a5c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:32:46.513027 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e20c35-1459-49df-901e-e5bc19429a5c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:32:46.514886 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e20c35-1459-49df-901e-e5bc19429a5c service neutron] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:32:46.533496 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 666/1331] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:32:46 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:32:48.651190 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7317984-1407-4a28-879d-dbfdd94810d2 None None] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:32:48.651747 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7317984-1407-4a28-879d-dbfdd94810d2 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:32:48.651747 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7317984-1407-4a28-879d-dbfdd94810d2 None None] PATH_INFO: `/` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:32:48.653104 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 666/1332] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:32:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:32:48.658396 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14334fb5-224b-411e-9f8a-65603ceb3d28 None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:32:48.658822 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14334fb5-224b-411e-9f8a-65603ceb3d28 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:32:48.658948 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14334fb5-224b-411e-9f8a-65603ceb3d28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:32:48.722124 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:32:48.724973 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-14334fb5-224b-411e-9f8a-65603ceb3d28 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:32:48.726815 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-14334fb5-224b-411e-9f8a-65603ceb3d28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:32:48.827770 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 667/1333] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:32:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:32:49.107807 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7317984-1407-4a28-879d-dbfdd94810d2 None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:32:49.109367 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-a7317984-1407-4a28-879d-dbfdd94810d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:32:49.209476 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7317984-1407-4a28-879d-dbfdd94810d2 None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:32:49.280341 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b3f024-97d9-43f3-8701-c621ef2b0d46 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43ad98a250a144099ccd18378aedd4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:32:49.280961 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b3f024-97d9-43f3-8701-c621ef2b0d46 service neutron] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:32:49.281241 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b3f024-97d9-43f3-8701-c621ef2b0d46 service neutron] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:32:49.281513 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b3f024-97d9-43f3-8701-c621ef2b0d46 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:32:49.283080 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-19b3f024-97d9-43f3-8701-c621ef2b0d46 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:32:49.308357 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:32:49.387641 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b3f024-97d9-43f3-8701-c621ef2b0d46 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:32:49.389540 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b3f024-97d9-43f3-8701-c621ef2b0d46 service neutron] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:32:49.407371 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 667/1334] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:32:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:32:51.332418 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c25129f-c0f9-4162-826e-ca4413091180 None None] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:32:51.333003 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c25129f-c0f9-4162-826e-ca4413091180 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:32:51.333003 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c25129f-c0f9-4162-826e-ca4413091180 None None] PATH_INFO: `/` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:32:51.334331 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 668/1335] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:32:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:32:51.339829 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96fc89b-2a11-44a0-98d0-b85a92db6d5b None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:32:51.340260 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96fc89b-2a11-44a0-98d0-b85a92db6d5b None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:32:51.340392 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96fc89b-2a11-44a0-98d0-b85a92db6d5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:32:51.405330 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-f96fc89b-2a11-44a0-98d0-b85a92db6d5b None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:32:51.406536 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-f96fc89b-2a11-44a0-98d0-b85a92db6d5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:32:51.478107 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:32:51.505209 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 668/1336] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:32:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:32:51.789003 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c25129f-c0f9-4162-826e-ca4413091180 None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:32:51.790549 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-5c25129f-c0f9-4162-826e-ca4413091180 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:32:51.891090 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c25129f-c0f9-4162-826e-ca4413091180 None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:32:51.918546 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:32:51.962256 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fbe993-b6a6-49cf-836c-1c1521407a69 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43ad98a250a144099ccd18378aedd4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:32:51.962853 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fbe993-b6a6-49cf-836c-1c1521407a69 service neutron] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:32:51.963130 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fbe993-b6a6-49cf-836c-1c1521407a69 service neutron] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:32:51.963404 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fbe993-b6a6-49cf-836c-1c1521407a69 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:32:51.965008 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-41fbe993-b6a6-49cf-836c-1c1521407a69 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:32:52.070732 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fbe993-b6a6-49cf-836c-1c1521407a69 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:32:52.072082 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fbe993-b6a6-49cf-836c-1c1521407a69 service neutron] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:32:52.089401 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 669/1337] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:32:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:32:54.285762 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50bae7b-4f07-4d84-9b85-f47557175a81 None None] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:32:54.286295 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50bae7b-4f07-4d84-9b85-f47557175a81 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:32:54.286295 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50bae7b-4f07-4d84-9b85-f47557175a81 None None] PATH_INFO: `/` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:32:54.287636 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 669/1338] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:32:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:32:54.292846 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2407ea8-c5c8-4815-bb43-128e95805870 None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:32:54.293286 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2407ea8-c5c8-4815-bb43-128e95805870 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:32:54.293398 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2407ea8-c5c8-4815-bb43-128e95805870 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:32:54.357599 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-b2407ea8-c5c8-4815-bb43-128e95805870 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:32:54.358707 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-b2407ea8-c5c8-4815-bb43-128e95805870 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:32:54.366878 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:32:54.455519 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 670/1339] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:32:54 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:32:54.733156 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50bae7b-4f07-4d84-9b85-f47557175a81 None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:32:54.734681 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-a50bae7b-4f07-4d84-9b85-f47557175a81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:32:54.835517 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50bae7b-4f07-4d84-9b85-f47557175a81 None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:32:54.906031 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1a0dcd-8a4f-48af-b77c-331b51213c02 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43ad98a250a144099ccd18378aedd4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:32:54.906627 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1a0dcd-8a4f-48af-b77c-331b51213c02 service neutron] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:32:54.906906 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1a0dcd-8a4f-48af-b77c-331b51213c02 service neutron] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:32:54.907180 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1a0dcd-8a4f-48af-b77c-331b51213c02 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:32:54.908779 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-aa1a0dcd-8a4f-48af-b77c-331b51213c02 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:32:54.943020 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:32:55.014191 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1a0dcd-8a4f-48af-b77c-331b51213c02 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:32:55.015629 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1a0dcd-8a4f-48af-b77c-331b51213c02 service neutron] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:32:55.033539 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 670/1340] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:32:54 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:32:58.771660 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585ccb83-7b5e-4133-91a5-38a311cd1944 None None] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:32:58.772189 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585ccb83-7b5e-4133-91a5-38a311cd1944 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:32:58.772189 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585ccb83-7b5e-4133-91a5-38a311cd1944 None None] PATH_INFO: `/` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:32:58.773720 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 671/1341] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:32:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:32:58.778878 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810133e3-1565-4a63-baa1-6d04d73358bc None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:32:58.779302 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810133e3-1565-4a63-baa1-6d04d73358bc None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:32:58.779440 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810133e3-1565-4a63-baa1-6d04d73358bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:32:58.849634 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-810133e3-1565-4a63-baa1-6d04d73358bc None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:32:58.850975 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-810133e3-1565-4a63-baa1-6d04d73358bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:32:58.937478 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:32:58.948923 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 671/1342] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:32:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:32:59.234148 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585ccb83-7b5e-4133-91a5-38a311cd1944 None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:32:59.235683 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-585ccb83-7b5e-4133-91a5-38a311cd1944 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:32:59.339773 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585ccb83-7b5e-4133-91a5-38a311cd1944 None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:32:59.378761 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:32:59.421020 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3e3369-ace5-417e-ac9b-c6f072a07f56 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43ad98a250a144099ccd18378aedd4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:32:59.421662 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3e3369-ace5-417e-ac9b-c6f072a07f56 service neutron] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:32:59.421946 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3e3369-ace5-417e-ac9b-c6f072a07f56 service neutron] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:32:59.422172 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3e3369-ace5-417e-ac9b-c6f072a07f56 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:32:59.423776 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-ee3e3369-ace5-417e-ac9b-c6f072a07f56 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:32:59.531079 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3e3369-ace5-417e-ac9b-c6f072a07f56 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:32:59.532428 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3e3369-ace5-417e-ac9b-c6f072a07f56 service neutron] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:32:59.550649 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 672/1343] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:32:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:33:02.090770 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae256a21-0353-4e70-962e-3db3190c208c None None] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:33:02.091265 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae256a21-0353-4e70-962e-3db3190c208c None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:33:02.091265 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae256a21-0353-4e70-962e-3db3190c208c None None] PATH_INFO: `/` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:33:02.092645 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 672/1344] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:33:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:33:02.097742 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be85952-c1b3-4701-94ec-3aab3a76925d None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:33:02.098177 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be85952-c1b3-4701-94ec-3aab3a76925d None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:33:02.098291 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be85952-c1b3-4701-94ec-3aab3a76925d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:33:02.165190 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-5be85952-c1b3-4701-94ec-3aab3a76925d None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:33:02.166397 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-5be85952-c1b3-4701-94ec-3aab3a76925d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:33:02.185328 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:33:02.264719 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 673/1345] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:33:02 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:33:02.558991 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae256a21-0353-4e70-962e-3db3190c208c None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:33:02.560563 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-ae256a21-0353-4e70-962e-3db3190c208c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:33:02.664343 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae256a21-0353-4e70-962e-3db3190c208c None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:33:02.741343 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716b5cac-e109-427f-957c-009d4a80dccc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43ad98a250a144099ccd18378aedd4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:33:02.742225 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716b5cac-e109-427f-957c-009d4a80dccc service neutron] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:33:02.742506 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716b5cac-e109-427f-957c-009d4a80dccc service neutron] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:33:02.742781 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716b5cac-e109-427f-957c-009d4a80dccc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:33:02.744577 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-716b5cac-e109-427f-957c-009d4a80dccc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:33:02.787668 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:33:02.852829 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716b5cac-e109-427f-957c-009d4a80dccc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:33:02.854186 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716b5cac-e109-427f-957c-009d4a80dccc service neutron] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:33:02.872775 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 673/1346] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:33:02 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:33:06.709971 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb707edf-4239-4f6e-9e0d-33baffcc24e9 None None] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:33:06.710453 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb707edf-4239-4f6e-9e0d-33baffcc24e9 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:33:06.710632 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb707edf-4239-4f6e-9e0d-33baffcc24e9 None None] PATH_INFO: `/` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:33:06.712053 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 674/1347] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:33:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:33:06.717469 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b9ae38-5b6f-4710-8102-d4683b0e14a6 None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:33:06.717978 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b9ae38-5b6f-4710-8102-d4683b0e14a6 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:33:06.718084 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b9ae38-5b6f-4710-8102-d4683b0e14a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:33:06.797248 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-96b9ae38-5b6f-4710-8102-d4683b0e14a6 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:33:06.798844 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-96b9ae38-5b6f-4710-8102-d4683b0e14a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:33:06.906551 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:33:06.910479 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 674/1348] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:33:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:33:07.196182 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb707edf-4239-4f6e-9e0d-33baffcc24e9 None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:33:07.197735 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-cb707edf-4239-4f6e-9e0d-33baffcc24e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:33:07.302464 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb707edf-4239-4f6e-9e0d-33baffcc24e9 None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:33:07.352542 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:33:07.378813 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d04733-cfcc-4b67-9fd3-870a55ed9a42 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43ad98a250a144099ccd18378aedd4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:33:07.379421 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d04733-cfcc-4b67-9fd3-870a55ed9a42 service neutron] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:33:07.379696 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d04733-cfcc-4b67-9fd3-870a55ed9a42 service neutron] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:33:07.379966 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d04733-cfcc-4b67-9fd3-870a55ed9a42 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:33:07.381573 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-a2d04733-cfcc-4b67-9fd3-870a55ed9a42 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:33:07.488829 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d04733-cfcc-4b67-9fd3-870a55ed9a42 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:33:07.490438 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d04733-cfcc-4b67-9fd3-870a55ed9a42 service neutron] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:33:07.508966 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 675/1349] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:33:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:33:12.423989 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3b5362-5059-41ce-92f3-d778ae93e717 None None] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:33:12.424514 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3b5362-5059-41ce-92f3-d778ae93e717 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:33:12.425092 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3b5362-5059-41ce-92f3-d778ae93e717 None None] PATH_INFO: `/` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:33:12.427273 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 675/1350] 10.222.0.45 () {54 vars in 921 bytes} [Wed Aug 24 17:33:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:33:12.432931 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c8a54f-0bbd-4b9f-b9a4-a29c16e05d6f None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:33:12.433871 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c8a54f-0bbd-4b9f-b9a4-a29c16e05d6f None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:33:12.433871 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c8a54f-0bbd-4b9f-b9a4-a29c16e05d6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:33:12.550721 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-18c8a54f-0bbd-4b9f-b9a4-a29c16e05d6f None None] MFA Rules not processed for user `37def4721b554b9ebb7704a19642d15b`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:33:12.551847 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-18c8a54f-0bbd-4b9f-b9a4-a29c16e05d6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:33:12.575025 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:33:12.696362 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 676/1351] 10.222.0.45 () {58 vars in 1018 bytes} [Wed Aug 24 17:33:12 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:33:16.798665 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1cb8d7-4535-4ef0-a457-6a97a136b252 None None] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:33:16.799860 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1cb8d7-4535-4ef0-a457-6a97a136b252 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:33:16.800691 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1cb8d7-4535-4ef0-a457-6a97a136b252 None None] PATH_INFO: `/` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:33:16.802621 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 676/1352] 10.222.0.45 () {54 vars in 921 bytes} [Wed Aug 24 17:33:16 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:33:16.808036 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bba7ba-cd97-4c08-9a78-85bb20e5e21e None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:33:16.808472 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bba7ba-cd97-4c08-9a78-85bb20e5e21e None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:33:16.808745 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bba7ba-cd97-4c08-9a78-85bb20e5e21e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:33:16.886324 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-76bba7ba-cd97-4c08-9a78-85bb20e5e21e None None] MFA Rules not processed for user `37def4721b554b9ebb7704a19642d15b`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:33:16.887540 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-76bba7ba-cd97-4c08-9a78-85bb20e5e21e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:33:16.962179 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:33:17.000978 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 677/1353] 10.222.0.45 () {58 vars in 1018 bytes} [Wed Aug 24 17:33:16 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:33:18.324830 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5992fab8-794e-497e-b418-2574673dd9ad None None] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:33:18.325419 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5992fab8-794e-497e-b418-2574673dd9ad None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:33:18.325419 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5992fab8-794e-497e-b418-2574673dd9ad None None] PATH_INFO: `/` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:33:18.326763 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 677/1354] 10.222.0.45 () {54 vars in 921 bytes} [Wed Aug 24 17:33:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:33:18.331963 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552847ff-09e9-4071-b9a7-44d772df284a None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:33:18.332441 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552847ff-09e9-4071-b9a7-44d772df284a None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:33:18.332690 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552847ff-09e9-4071-b9a7-44d772df284a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:33:18.408420 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-552847ff-09e9-4071-b9a7-44d772df284a None None] MFA Rules not processed for user `37def4721b554b9ebb7704a19642d15b`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:33:18.409647 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-552847ff-09e9-4071-b9a7-44d772df284a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:33:18.514129 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 678/1355] 10.222.0.45 () {58 vars in 1018 bytes} [Wed Aug 24 17:33:18 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:33:21.812935 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d0914e-83ef-499d-a19e-af73cb1ae341 None None] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:33:21.813488 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d0914e-83ef-499d-a19e-af73cb1ae341 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:33:21.813588 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d0914e-83ef-499d-a19e-af73cb1ae341 None None] PATH_INFO: `/` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:33:21.814893 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 678/1356] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:33:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:33:21.820255 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9433e4-0c97-4c26-b40b-bf5bcdaf48c5 None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:33:21.820677 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9433e4-0c97-4c26-b40b-bf5bcdaf48c5 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:33:21.820844 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9433e4-0c97-4c26-b40b-bf5bcdaf48c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:33:21.869055 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:33:21.904801 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-5c9433e4-0c97-4c26-b40b-bf5bcdaf48c5 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:33:21.905980 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-5c9433e4-0c97-4c26-b40b-bf5bcdaf48c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:33:22.018407 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 679/1357] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:33:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:33:22.320755 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba005bc1-0bdc-4257-847b-66d88478b001 None None] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:33:22.321711 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba005bc1-0bdc-4257-847b-66d88478b001 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:33:22.322249 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba005bc1-0bdc-4257-847b-66d88478b001 None None] PATH_INFO: `/` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:33:22.324338 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 679/1358] 10.222.0.45 () {54 vars in 962 bytes} [Wed Aug 24 17:33:22 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 17:33:22.332808 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f18c2d-1fc0-47eb-b28d-9d5d2c537f38 None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:33:22.333520 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f18c2d-1fc0-47eb-b28d-9d5d2c537f38 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:33:22.333520 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f18c2d-1fc0-47eb-b28d-9d5d2c537f38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:33:22.415672 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-45f18c2d-1fc0-47eb-b28d-9d5d2c537f38 None None] MFA Rules not processed for user `053b7ce6d6f649ed91a26e434dab805e`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:33:22.417457 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-45f18c2d-1fc0-47eb-b28d-9d5d2c537f38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:33:22.468831 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:33:22.568905 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 680/1359] 10.222.0.45 () {58 vars in 1059 bytes} [Wed Aug 24 17:33:22 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:33:22.576501 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba005bc1-0bdc-4257-847b-66d88478b001 None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:33:22.578145 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-ba005bc1-0bdc-4257-847b-66d88478b001 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:33:22.739621 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba005bc1-0bdc-4257-847b-66d88478b001 None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:33:22.819682 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:33:22.862299 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378e637f-5538-4ff2-ab65-f42c8299e6c0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '053b7ce6d6f649ed91a26e434dab805e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:33:22.862913 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378e637f-5538-4ff2-ab65-f42c8299e6c0 service nova] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:33:22.863193 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378e637f-5538-4ff2-ab65-f42c8299e6c0 service nova] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:33:22.863464 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378e637f-5538-4ff2-ab65-f42c8299e6c0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:33:22.865162 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-378e637f-5538-4ff2-ab65-f42c8299e6c0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:33:22.955882 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378e637f-5538-4ff2-ab65-f42c8299e6c0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:33:22.957519 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378e637f-5538-4ff2-ab65-f42c8299e6c0 service nova] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:33:22.976212 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 680/1360] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:33:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 17:33:25.102371 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ec6b40-6323-465c-96f7-5b5d722a2560 None None] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:33:25.102804 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ec6b40-6323-465c-96f7-5b5d722a2560 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:33:25.102988 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ec6b40-6323-465c-96f7-5b5d722a2560 None None] PATH_INFO: `/` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:33:25.108182 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 681/1361] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:33:25 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:33:25.110022 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f69f3b-35f8-47fc-95b1-5efa58b6c675 None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:33:25.110462 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f69f3b-35f8-47fc-95b1-5efa58b6c675 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:33:25.110882 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f69f3b-35f8-47fc-95b1-5efa58b6c675 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:33:25.182195 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-b3f69f3b-35f8-47fc-95b1-5efa58b6c675 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:33:25.183403 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-b3f69f3b-35f8-47fc-95b1-5efa58b6c675 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:33:25.223420 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:33:25.286385 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 681/1362] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:33:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:33:25.578871 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ec6b40-6323-465c-96f7-5b5d722a2560 None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:33:25.580458 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-e8ec6b40-6323-465c-96f7-5b5d722a2560 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:33:25.732684 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ec6b40-6323-465c-96f7-5b5d722a2560 None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:33:25.742328 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:33:25.857832 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fc4069-e57f-4bfd-885e-4567403f4947 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '053b7ce6d6f649ed91a26e434dab805e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:33:25.858362 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fc4069-e57f-4bfd-885e-4567403f4947 service nova] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:33:25.858638 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fc4069-e57f-4bfd-885e-4567403f4947 service nova] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:33:25.859095 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fc4069-e57f-4bfd-885e-4567403f4947 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:33:25.860738 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-40fc4069-e57f-4bfd-885e-4567403f4947 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:33:25.956621 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fc4069-e57f-4bfd-885e-4567403f4947 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:33:25.958248 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fc4069-e57f-4bfd-885e-4567403f4947 service nova] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:33:25.988863 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 682/1363] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:33:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:33:28.215256 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba53a82-ba74-4bee-8eac-a127fc277483 None None] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:33:28.215800 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba53a82-ba74-4bee-8eac-a127fc277483 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:33:28.215800 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba53a82-ba74-4bee-8eac-a127fc277483 None None] PATH_INFO: `/` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:33:28.217786 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 682/1364] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:33:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:33:28.223542 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d63460-e6e4-4457-8f4e-6573cb203ed2 None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:33:28.223985 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d63460-e6e4-4457-8f4e-6573cb203ed2 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:33:28.224065 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d63460-e6e4-4457-8f4e-6573cb203ed2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:33:28.246916 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:33:28.304416 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-89d63460-e6e4-4457-8f4e-6573cb203ed2 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:33:28.305820 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-89d63460-e6e4-4457-8f4e-6573cb203ed2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:33:28.420939 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 683/1365] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:33:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:33:28.712976 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6aa778c-f7db-4400-8c81-537e026af239 None None] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:33:28.713266 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6aa778c-f7db-4400-8c81-537e026af239 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:33:28.713844 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6aa778c-f7db-4400-8c81-537e026af239 None None] PATH_INFO: `/` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:33:28.715363 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 683/1366] 10.222.0.45 () {54 vars in 962 bytes} [Wed Aug 24 17:33:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:33:28.722875 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574040eb-061c-44c7-924e-7d4a9aacf291 None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:33:28.723360 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574040eb-061c-44c7-924e-7d4a9aacf291 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:33:28.723850 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574040eb-061c-44c7-924e-7d4a9aacf291 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:33:28.815950 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-574040eb-061c-44c7-924e-7d4a9aacf291 None None] MFA Rules not processed for user `053b7ce6d6f649ed91a26e434dab805e`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:33:28.817206 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-574040eb-061c-44c7-924e-7d4a9aacf291 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:33:28.857366 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:33:28.956828 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 684/1367] 10.222.0.45 () {58 vars in 1059 bytes} [Wed Aug 24 17:33:28 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:33:28.960248 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6aa778c-f7db-4400-8c81-537e026af239 None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:33:28.961880 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-c6aa778c-f7db-4400-8c81-537e026af239 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:33:29.139378 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6aa778c-f7db-4400-8c81-537e026af239 None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:33:29.160512 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:33:29.265221 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ade6816-18d1-4fd4-b680-e31e0f22bdc6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '053b7ce6d6f649ed91a26e434dab805e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:33:29.266554 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ade6816-18d1-4fd4-b680-e31e0f22bdc6 service nova] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:33:29.267038 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ade6816-18d1-4fd4-b680-e31e0f22bdc6 service nova] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:33:29.267503 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ade6816-18d1-4fd4-b680-e31e0f22bdc6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:33:29.269320 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-2ade6816-18d1-4fd4-b680-e31e0f22bdc6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:33:29.361346 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ade6816-18d1-4fd4-b680-e31e0f22bdc6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:33:29.363305 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ade6816-18d1-4fd4-b680-e31e0f22bdc6 service nova] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:33:29.382175 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 684/1368] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:33:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:33:30.021352 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3139424d-b00b-449d-a24b-612a257097df None None] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:33:30.021903 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3139424d-b00b-449d-a24b-612a257097df None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:33:30.021903 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3139424d-b00b-449d-a24b-612a257097df None None] PATH_INFO: `/` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:33:30.024666 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 685/1369] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:33:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:33:30.028652 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfeabd2-979d-4035-9a2c-3a224c69ec07 None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:33:30.029446 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfeabd2-979d-4035-9a2c-3a224c69ec07 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:33:30.029733 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfeabd2-979d-4035-9a2c-3a224c69ec07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:33:30.063885 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:33:30.100684 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-fdfeabd2-979d-4035-9a2c-3a224c69ec07 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:33:30.101898 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-fdfeabd2-979d-4035-9a2c-3a224c69ec07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:33:30.204585 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 685/1370] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:33:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:33:30.791567 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9835d26-53f0-4559-89ae-abcb8fe06a63 None None] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:33:30.791567 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9835d26-53f0-4559-89ae-abcb8fe06a63 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:33:30.791567 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9835d26-53f0-4559-89ae-abcb8fe06a63 None None] PATH_INFO: `/` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:33:30.791567 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 686/1371] 10.222.0.45 () {54 vars in 964 bytes} [Wed Aug 24 17:33:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:33:30.800776 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17a3c68-99d6-4628-bfea-7dc73a44fe9f None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:33:30.800776 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17a3c68-99d6-4628-bfea-7dc73a44fe9f None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:33:30.800776 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17a3c68-99d6-4628-bfea-7dc73a44fe9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:33:30.871812 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-a17a3c68-99d6-4628-bfea-7dc73a44fe9f None None] MFA Rules not processed for user `59be342541264942ad87f60ff4de6c8e`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:33:30.872980 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-a17a3c68-99d6-4628-bfea-7dc73a44fe9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:33:30.920390 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:33:30.954888 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 686/1372] 10.222.0.45 () {58 vars in 1061 bytes} [Wed Aug 24 17:33:30 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:33:30.961659 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9835d26-53f0-4559-89ae-abcb8fe06a63 None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:33:30.963352 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-d9835d26-53f0-4559-89ae-abcb8fe06a63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:33:31.075773 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9835d26-53f0-4559-89ae-abcb8fe06a63 None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:33:31.121437 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:33:31.165434 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6507c9-628d-40a6-af0d-c8c94914bf71 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59be342541264942ad87f60ff4de6c8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:33:31.166490 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6507c9-628d-40a6-af0d-c8c94914bf71 service cinder] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:33:31.166947 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6507c9-628d-40a6-af0d-c8c94914bf71 service cinder] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:33:31.167398 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6507c9-628d-40a6-af0d-c8c94914bf71 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:33:31.169201 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-4e6507c9-628d-40a6-af0d-c8c94914bf71 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:33:31.298379 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6507c9-628d-40a6-af0d-c8c94914bf71 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:33:31.299872 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6507c9-628d-40a6-af0d-c8c94914bf71 service cinder] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:33:31.320008 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 687/1373] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:33:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:33:31.362906 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39bbbef-a751-45a5-82b1-5ab8583679cb None None] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:33:31.363208 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39bbbef-a751-45a5-82b1-5ab8583679cb None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:33:31.363485 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39bbbef-a751-45a5-82b1-5ab8583679cb None None] PATH_INFO: `/` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:33:31.365125 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 687/1374] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:33:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:33:31.370421 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594f044e-e1a9-41c2-a611-411dcd4cb3a0 None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:33:31.370776 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594f044e-e1a9-41c2-a611-411dcd4cb3a0 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:33:31.370973 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594f044e-e1a9-41c2-a611-411dcd4cb3a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:33:31.447716 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-594f044e-e1a9-41c2-a611-411dcd4cb3a0 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:33:31.449451 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-594f044e-e1a9-41c2-a611-411dcd4cb3a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:33:31.473545 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:33:31.572898 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 688/1375] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:33:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:33:31.860801 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39bbbef-a751-45a5-82b1-5ab8583679cb None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:33:31.862452 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-b39bbbef-a751-45a5-82b1-5ab8583679cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:33:32.011289 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39bbbef-a751-45a5-82b1-5ab8583679cb None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:33:32.062840 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:33:32.125109 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2a9c1e-6ced-4392-9867-71db4bfb1e02 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '053b7ce6d6f649ed91a26e434dab805e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:33:32.125718 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2a9c1e-6ced-4392-9867-71db4bfb1e02 service nova] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:33:32.125992 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2a9c1e-6ced-4392-9867-71db4bfb1e02 service nova] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:33:32.126292 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2a9c1e-6ced-4392-9867-71db4bfb1e02 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:33:32.127912 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-9a2a9c1e-6ced-4392-9867-71db4bfb1e02 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:33:32.217938 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2a9c1e-6ced-4392-9867-71db4bfb1e02 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:33:32.219543 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2a9c1e-6ced-4392-9867-71db4bfb1e02 service nova] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:33:32.238154 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 688/1376] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:33:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:33:34.157385 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd31df86-2509-4936-b465-e5c88f23f7e2 None None] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:33:34.157897 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd31df86-2509-4936-b465-e5c88f23f7e2 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:33:34.158018 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd31df86-2509-4936-b465-e5c88f23f7e2 None None] PATH_INFO: `/` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:33:34.159256 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 689/1377] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:33:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:33:34.164607 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8efd51-86ce-42a9-b31c-8060a6cbd954 None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:33:34.165021 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8efd51-86ce-42a9-b31c-8060a6cbd954 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:33:34.165155 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8efd51-86ce-42a9-b31c-8060a6cbd954 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:33:34.239111 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-9d8efd51-86ce-42a9-b31c-8060a6cbd954 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:33:34.240310 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-9d8efd51-86ce-42a9-b31c-8060a6cbd954 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:33:34.258817 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:33:34.351730 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 689/1378] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:33:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:33:34.643220 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd31df86-2509-4936-b465-e5c88f23f7e2 None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:33:34.645266 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-bd31df86-2509-4936-b465-e5c88f23f7e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:33:34.857371 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd31df86-2509-4936-b465-e5c88f23f7e2 None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:33:34.863709 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:33:34.988302 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b563e7f0-9a61-4dd9-afc3-6f6113654d37 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '053b7ce6d6f649ed91a26e434dab805e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:33:34.988941 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b563e7f0-9a61-4dd9-afc3-6f6113654d37 service nova] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:33:34.989334 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b563e7f0-9a61-4dd9-afc3-6f6113654d37 service nova] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:33:34.989614 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b563e7f0-9a61-4dd9-afc3-6f6113654d37 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:33:34.991441 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-b563e7f0-9a61-4dd9-afc3-6f6113654d37 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:33:35.088714 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b563e7f0-9a61-4dd9-afc3-6f6113654d37 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:33:35.090439 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b563e7f0-9a61-4dd9-afc3-6f6113654d37 service nova] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:33:35.110181 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 690/1379] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:33:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 467 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:33:37.050047 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b28e9e0-b560-4473-bdd8-57c47668a492 None None] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:33:37.050552 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b28e9e0-b560-4473-bdd8-57c47668a492 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:33:37.050635 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b28e9e0-b560-4473-bdd8-57c47668a492 None None] PATH_INFO: `/` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:33:37.052101 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 690/1380] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:33:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:33:37.057204 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f904422-98db-47c9-b324-b92b087edc74 None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:33:37.057594 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f904422-98db-47c9-b324-b92b087edc74 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:33:37.057774 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f904422-98db-47c9-b324-b92b087edc74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:33:37.069749 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:33:37.133936 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-7f904422-98db-47c9-b324-b92b087edc74 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:33:37.135123 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-7f904422-98db-47c9-b324-b92b087edc74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:33:37.239757 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 691/1381] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:33:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:33:37.525086 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b28e9e0-b560-4473-bdd8-57c47668a492 None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:33:37.526891 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-1b28e9e0-b560-4473-bdd8-57c47668a492 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:33:37.683248 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:33:37.694755 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b28e9e0-b560-4473-bdd8-57c47668a492 None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:33:37.822829 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa7f7fd-daed-4c51-ae7c-c46a64a905ef service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '053b7ce6d6f649ed91a26e434dab805e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:33:37.823562 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa7f7fd-daed-4c51-ae7c-c46a64a905ef service nova] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:33:37.823917 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa7f7fd-daed-4c51-ae7c-c46a64a905ef service nova] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:33:37.824393 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa7f7fd-daed-4c51-ae7c-c46a64a905ef service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:33:37.826211 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-9aa7f7fd-daed-4c51-ae7c-c46a64a905ef service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:33:37.926169 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa7f7fd-daed-4c51-ae7c-c46a64a905ef service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:33:37.927893 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa7f7fd-daed-4c51-ae7c-c46a64a905ef service nova] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:33:37.939230 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:33:37.949076 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 691/1382] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:33:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:33:39.986256 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13536a54-3430-44e4-a6b1-5673a07ba24d None None] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:33:39.987024 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13536a54-3430-44e4-a6b1-5673a07ba24d None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:33:39.987024 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13536a54-3430-44e4-a6b1-5673a07ba24d None None] PATH_INFO: `/` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:33:39.988299 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 692/1383] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:33:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:33:39.994374 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6bc6ef-2b85-460f-b3fa-ddc1b3d1e622 None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:33:39.994813 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6bc6ef-2b85-460f-b3fa-ddc1b3d1e622 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:33:39.995118 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6bc6ef-2b85-460f-b3fa-ddc1b3d1e622 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:33:40.079060 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-1f6bc6ef-2b85-460f-b3fa-ddc1b3d1e622 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:33:40.080753 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-1f6bc6ef-2b85-460f-b3fa-ddc1b3d1e622 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:33:40.191792 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 692/1384] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:33:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:33:40.482296 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13536a54-3430-44e4-a6b1-5673a07ba24d None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:33:40.483934 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-13536a54-3430-44e4-a6b1-5673a07ba24d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:33:40.587597 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:33:40.650589 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13536a54-3430-44e4-a6b1-5673a07ba24d None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:33:40.768842 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a23af1e-aa81-4cd5-b62b-e0302df4caa6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '053b7ce6d6f649ed91a26e434dab805e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:33:40.769373 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a23af1e-aa81-4cd5-b62b-e0302df4caa6 service nova] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:33:40.769651 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a23af1e-aa81-4cd5-b62b-e0302df4caa6 service nova] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:33:40.769925 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a23af1e-aa81-4cd5-b62b-e0302df4caa6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:33:40.771537 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-6a23af1e-aa81-4cd5-b62b-e0302df4caa6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:33:40.823948 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:33:40.864525 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a23af1e-aa81-4cd5-b62b-e0302df4caa6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:33:40.866175 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a23af1e-aa81-4cd5-b62b-e0302df4caa6 service nova] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:33:40.885316 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 693/1385] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:33:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:33:42.789018 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1460a1e0-641a-46ef-b1e2-4e6c702b361e None None] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:33:42.789540 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1460a1e0-641a-46ef-b1e2-4e6c702b361e None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:33:42.789645 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1460a1e0-641a-46ef-b1e2-4e6c702b361e None None] PATH_INFO: `/` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:33:42.790910 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 693/1386] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:33:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 17:33:42.798472 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61204350-ff1f-46f8-8bf8-5f9dfed43e68 None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:33:42.798917 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61204350-ff1f-46f8-8bf8-5f9dfed43e68 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:33:42.799004 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61204350-ff1f-46f8-8bf8-5f9dfed43e68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:33:42.866226 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-61204350-ff1f-46f8-8bf8-5f9dfed43e68 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:33:42.867447 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-61204350-ff1f-46f8-8bf8-5f9dfed43e68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:33:42.964013 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 694/1387] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:33:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:33:43.257183 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1460a1e0-641a-46ef-b1e2-4e6c702b361e None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:33:43.258840 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-1460a1e0-641a-46ef-b1e2-4e6c702b361e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:33:43.338087 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:33:43.407944 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1460a1e0-641a-46ef-b1e2-4e6c702b361e None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:33:43.533975 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fbbe31-8090-466a-a282-c615c6645de4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '053b7ce6d6f649ed91a26e434dab805e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:33:43.534939 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fbbe31-8090-466a-a282-c615c6645de4 service nova] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:33:43.535305 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fbbe31-8090-466a-a282-c615c6645de4 service nova] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:33:43.535660 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fbbe31-8090-466a-a282-c615c6645de4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:33:43.537889 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-d4fbbe31-8090-466a-a282-c615c6645de4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:33:43.586652 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:33:43.641774 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fbbe31-8090-466a-a282-c615c6645de4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:33:43.643985 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fbbe31-8090-466a-a282-c615c6645de4 service nova] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:33:43.667077 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 694/1388] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:33:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:33:45.587975 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e21ec31-e8e8-4eb7-9a2c-968715e74cb2 None None] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:33:45.588575 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e21ec31-e8e8-4eb7-9a2c-968715e74cb2 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:33:45.588575 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e21ec31-e8e8-4eb7-9a2c-968715e74cb2 None None] PATH_INFO: `/` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:33:45.590004 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 695/1389] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:33:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:33:45.595419 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa20ad6e-11a2-4816-9654-b384fac7e006 None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:33:45.595827 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa20ad6e-11a2-4816-9654-b384fac7e006 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:33:45.596070 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa20ad6e-11a2-4816-9654-b384fac7e006 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:33:45.665302 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-fa20ad6e-11a2-4816-9654-b384fac7e006 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:33:45.666511 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-fa20ad6e-11a2-4816-9654-b384fac7e006 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:33:45.760952 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 695/1390] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:33:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:33:46.049224 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e21ec31-e8e8-4eb7-9a2c-968715e74cb2 None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:33:46.050818 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-9e21ec31-e8e8-4eb7-9a2c-968715e74cb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:33:46.082074 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:33:46.200357 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e21ec31-e8e8-4eb7-9a2c-968715e74cb2 None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:33:46.302378 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:33:46.315515 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725d4378-a1a6-45a6-bf0a-5640f7f7081a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '053b7ce6d6f649ed91a26e434dab805e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:33:46.316124 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725d4378-a1a6-45a6-bf0a-5640f7f7081a service nova] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:33:46.316402 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725d4378-a1a6-45a6-bf0a-5640f7f7081a service nova] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:33:46.316712 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725d4378-a1a6-45a6-bf0a-5640f7f7081a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:33:46.318303 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-725d4378-a1a6-45a6-bf0a-5640f7f7081a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:33:46.405969 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725d4378-a1a6-45a6-bf0a-5640f7f7081a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:33:46.407569 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725d4378-a1a6-45a6-bf0a-5640f7f7081a service nova] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:33:46.426685 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 696/1391] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:33:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:33:46.455844 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603353ce-c5b0-4acb-974d-ef9b6b2126e4 None None] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:33:46.456144 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603353ce-c5b0-4acb-974d-ef9b6b2126e4 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:33:46.456412 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603353ce-c5b0-4acb-974d-ef9b6b2126e4 None None] PATH_INFO: `/` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:33:46.457885 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 696/1392] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:33:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:33:46.463229 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601c7062-676b-428b-9727-0f562e73faf9 None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:33:46.463418 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601c7062-676b-428b-9727-0f562e73faf9 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:33:46.463686 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601c7062-676b-428b-9727-0f562e73faf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:33:46.541044 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-601c7062-676b-428b-9727-0f562e73faf9 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:33:46.542344 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-601c7062-676b-428b-9727-0f562e73faf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:33:46.606169 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:33:46.646973 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 697/1393] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:33:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:33:46.945852 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a110a5e4-e1e9-4a06-a1d1-fa926858d319 None None] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:33:46.946381 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a110a5e4-e1e9-4a06-a1d1-fa926858d319 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:33:46.946381 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a110a5e4-e1e9-4a06-a1d1-fa926858d319 None None] PATH_INFO: `/` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:33:46.947704 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 697/1394] 10.222.0.45 () {54 vars in 967 bytes} [Wed Aug 24 17:33:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:33:46.953217 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2405b3e-e43a-48a0-b9c9-a3ce6a2bd097 None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:33:46.953497 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2405b3e-e43a-48a0-b9c9-a3ce6a2bd097 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:33:46.953764 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2405b3e-e43a-48a0-b9c9-a3ce6a2bd097 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:33:47.027473 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-c2405b3e-e43a-48a0-b9c9-a3ce6a2bd097 None None] MFA Rules not processed for user `df5fee57539447428dd1c72eab3cfd9d`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:33:47.028680 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-c2405b3e-e43a-48a0-b9c9-a3ce6a2bd097 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:33:47.103767 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 698/1395] 10.222.0.45 () {58 vars in 1064 bytes} [Wed Aug 24 17:33:46 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:33:47.111455 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a110a5e4-e1e9-4a06-a1d1-fa926858d319 None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:33:47.112960 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-a110a5e4-e1e9-4a06-a1d1-fa926858d319 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:33:47.126726 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:33:47.216534 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a110a5e4-e1e9-4a06-a1d1-fa926858d319 None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:33:47.289025 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d020c6-a5e9-4fc7-a8eb-37b444d837db service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df5fee57539447428dd1c72eab3cfd9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:33:47.289631 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d020c6-a5e9-4fc7-a8eb-37b444d837db service glance] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:33:47.289908 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d020c6-a5e9-4fc7-a8eb-37b444d837db service glance] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:33:47.290178 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d020c6-a5e9-4fc7-a8eb-37b444d837db service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:33:47.291765 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-31d020c6-a5e9-4fc7-a8eb-37b444d837db service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:33:47.358343 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:33:47.400142 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d020c6-a5e9-4fc7-a8eb-37b444d837db service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:33:47.401618 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d020c6-a5e9-4fc7-a8eb-37b444d837db service glance] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:33:47.419063 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 698/1396] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:33:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:33:48.153018 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ad516e-be40-49d0-b21b-368a4852a12a None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:33:48.153520 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ad516e-be40-49d0-b21b-368a4852a12a None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:33:48.153520 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ad516e-be40-49d0-b21b-368a4852a12a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:33:48.224631 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-c2ad516e-be40-49d0-b21b-368a4852a12a None None] MFA Rules not processed for user `ea143d042ee94ae7a4544d90d18db7db`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:33:48.225924 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-c2ad516e-be40-49d0-b21b-368a4852a12a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:33:48.235884 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:33:48.314167 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 699/1397] 10.222.0.45 () {58 vars in 1009 bytes} [Wed Aug 24 17:33:48 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:33:48.356826 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161586f1-7cce-4073-9c6e-989496e25a5c None None] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:33:48.357955 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161586f1-7cce-4073-9c6e-989496e25a5c None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:33:48.357955 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161586f1-7cce-4073-9c6e-989496e25a5c None None] PATH_INFO: `/` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:33:48.359303 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 699/1398] 10.222.0.45 () {54 vars in 943 bytes} [Wed Aug 24 17:33:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:33:48.365562 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11044716-0983-4cab-975b-519d7a0df0a2 None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:33:48.365851 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11044716-0983-4cab-975b-519d7a0df0a2 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:33:48.366127 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11044716-0983-4cab-975b-519d7a0df0a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:33:48.369214 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2a6819-e8a8-4c94-93a2-e221d3496cfb None None] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:33:48.369493 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2a6819-e8a8-4c94-93a2-e221d3496cfb None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:33:48.369764 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2a6819-e8a8-4c94-93a2-e221d3496cfb None None] PATH_INFO: `/` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:33:48.371010 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 700/1399] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:33:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 17:33:48.376082 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2285371f-cd37-4a71-8f34-3ae5c8aa54d4 None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:33:48.376369 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2285371f-cd37-4a71-8f34-3ae5c8aa54d4 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:33:48.376698 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2285371f-cd37-4a71-8f34-3ae5c8aa54d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:33:48.442648 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-2285371f-cd37-4a71-8f34-3ae5c8aa54d4 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:33:48.442997 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-11044716-0983-4cab-975b-519d7a0df0a2 None None] MFA Rules not processed for user `90f0772b64c84faaa689e7272fa97898`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:33:48.443785 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-2285371f-cd37-4a71-8f34-3ae5c8aa54d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:33:48.444135 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-11044716-0983-4cab-975b-519d7a0df0a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:33:48.531563 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:33:48.541952 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 701/1400] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:33:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:33:48.555172 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 700/1401] 10.222.0.45 () {58 vars in 1040 bytes} [Wed Aug 24 17:33:48 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:33:48.564582 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2285371f-cd37-4a71-8f34-3ae5c8aa54d4 None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:33:48.565822 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-2285371f-cd37-4a71-8f34-3ae5c8aa54d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:33:48.591382 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:33:48.667908 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2285371f-cd37-4a71-8f34-3ae5c8aa54d4 None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:33:48.775387 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d7143b-b73e-4d9f-a2dd-a511a7d580d6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90f0772b64c84faaa689e7272fa97898', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:33:48.776002 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d7143b-b73e-4d9f-a2dd-a511a7d580d6 service swift] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:33:48.776285 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d7143b-b73e-4d9f-a2dd-a511a7d580d6 service swift] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:33:48.776588 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d7143b-b73e-4d9f-a2dd-a511a7d580d6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:33:48.778266 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-77d7143b-b73e-4d9f-a2dd-a511a7d580d6 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:33:48.828861 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:33:48.838571 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11044716-0983-4cab-975b-519d7a0df0a2 None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:33:48.839844 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-11044716-0983-4cab-975b-519d7a0df0a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:33:48.871542 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d7143b-b73e-4d9f-a2dd-a511a7d580d6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:33:48.873728 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d7143b-b73e-4d9f-a2dd-a511a7d580d6 service swift] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:33:48.881770 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 702/1402] 10.222.0.45 () {60 vars in 1384 bytes} [Wed Aug 24 17:33:48 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 24 17:33:48.941626 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11044716-0983-4cab-975b-519d7a0df0a2 None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:33:49.062909 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:33:49.070493 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa39c2c-2f3b-4085-b214-a58b8a33fa14 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '053b7ce6d6f649ed91a26e434dab805e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:33:49.071118 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa39c2c-2f3b-4085-b214-a58b8a33fa14 service nova] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:33:49.071396 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa39c2c-2f3b-4085-b214-a58b8a33fa14 service nova] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:33:49.071708 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa39c2c-2f3b-4085-b214-a58b8a33fa14 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:33:49.073343 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-3fa39c2c-2f3b-4085-b214-a58b8a33fa14 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:33:49.170448 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa39c2c-2f3b-4085-b214-a58b8a33fa14 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:33:49.172102 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa39c2c-2f3b-4085-b214-a58b8a33fa14 service nova] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:33:49.193698 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 701/1403] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:33:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:33:51.130360 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a929143-8354-49b4-9c3d-765494a5c869 None None] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:33:51.130888 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a929143-8354-49b4-9c3d-765494a5c869 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:33:51.130888 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a929143-8354-49b4-9c3d-765494a5c869 None None] PATH_INFO: `/` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:33:51.132177 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 703/1404] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:33:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:33:51.138185 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964326ee-83f2-43a8-a9e3-5b6123d54f61 None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:33:51.138185 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964326ee-83f2-43a8-a9e3-5b6123d54f61 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:33:51.138185 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964326ee-83f2-43a8-a9e3-5b6123d54f61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:33:51.211600 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-964326ee-83f2-43a8-a9e3-5b6123d54f61 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:33:51.212868 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-964326ee-83f2-43a8-a9e3-5b6123d54f61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:33:51.287634 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:33:51.328001 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 702/1405] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:33:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:33:51.616985 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a929143-8354-49b4-9c3d-765494a5c869 None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:33:51.618498 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-2a929143-8354-49b4-9c3d-765494a5c869 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:33:51.783138 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a929143-8354-49b4-9c3d-765494a5c869 None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:33:51.821974 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:33:51.910561 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4ce508-b056-439b-8b93-9585a7ee6120 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '053b7ce6d6f649ed91a26e434dab805e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:33:51.911189 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4ce508-b056-439b-8b93-9585a7ee6120 service nova] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:33:51.911475 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4ce508-b056-439b-8b93-9585a7ee6120 service nova] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:33:51.911748 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4ce508-b056-439b-8b93-9585a7ee6120 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:33:51.913401 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-ae4ce508-b056-439b-8b93-9585a7ee6120 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:33:52.037810 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4ce508-b056-439b-8b93-9585a7ee6120 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:33:52.040228 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4ce508-b056-439b-8b93-9585a7ee6120 service nova] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:33:52.073134 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 704/1406] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:33:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 456 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:33:54.060303 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dea2a93-ea83-41c7-b4e6-8e8fa130ce38 None None] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:33:54.060909 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dea2a93-ea83-41c7-b4e6-8e8fa130ce38 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:33:54.060909 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dea2a93-ea83-41c7-b4e6-8e8fa130ce38 None None] PATH_INFO: `/` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:33:54.062441 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 703/1407] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:33:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:33:54.067489 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8e5abe-1de0-4d09-aaa7-acddffb20943 None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:33:54.067910 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8e5abe-1de0-4d09-aaa7-acddffb20943 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:33:54.068069 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8e5abe-1de0-4d09-aaa7-acddffb20943 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:33:54.138247 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:33:54.159686 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-1c8e5abe-1de0-4d09-aaa7-acddffb20943 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:33:54.160957 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-1c8e5abe-1de0-4d09-aaa7-acddffb20943 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:33:54.280577 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 705/1408] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:33:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:33:54.561722 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dea2a93-ea83-41c7-b4e6-8e8fa130ce38 None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:33:54.563291 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-6dea2a93-ea83-41c7-b4e6-8e8fa130ce38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:33:54.732822 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dea2a93-ea83-41c7-b4e6-8e8fa130ce38 None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:33:54.794907 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:33:54.873512 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23128f1-d81b-461b-b3ee-a3b3c44395c4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '053b7ce6d6f649ed91a26e434dab805e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:33:54.874795 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23128f1-d81b-461b-b3ee-a3b3c44395c4 service nova] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:33:54.875075 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23128f1-d81b-461b-b3ee-a3b3c44395c4 service nova] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:33:54.875343 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23128f1-d81b-461b-b3ee-a3b3c44395c4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:33:54.876979 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-e23128f1-d81b-461b-b3ee-a3b3c44395c4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:33:54.975984 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23128f1-d81b-461b-b3ee-a3b3c44395c4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:33:54.977938 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23128f1-d81b-461b-b3ee-a3b3c44395c4 service nova] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:33:54.996950 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 704/1409] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:33:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:34:04.267235 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=47 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:34:04.994419 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=75 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:34:10.861833 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139802f6-bc0e-4aa2-98e4-11ccf737876c None None] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:10.862359 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139802f6-bc0e-4aa2-98e4-11ccf737876c None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:10.862359 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139802f6-bc0e-4aa2-98e4-11ccf737876c None None] PATH_INFO: `/` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:10.863733 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 706/1410] 10.222.0.45 () {54 vars in 918 bytes} [Wed Aug 24 17:34:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:34:10.868738 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69632997-943a-4220-bbe5-89e88db70786 None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:10.869132 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69632997-943a-4220-bbe5-89e88db70786 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:10.869287 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69632997-943a-4220-bbe5-89e88db70786 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:10.943726 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-69632997-943a-4220-bbe5-89e88db70786 None None] MFA Rules not processed for user `37def4721b554b9ebb7704a19642d15b`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:34:10.944961 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-69632997-943a-4220-bbe5-89e88db70786 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:34:11.058365 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 705/1411] 10.222.0.45 () {58 vars in 1015 bytes} [Wed Aug 24 17:34:10 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:34:11.074769 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851c0a4b-8a15-40c8-aff2-4aa7d5081dcd None None] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:11.075049 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851c0a4b-8a15-40c8-aff2-4aa7d5081dcd None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:11.075321 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851c0a4b-8a15-40c8-aff2-4aa7d5081dcd None None] PATH_INFO: `/` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:11.077212 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 707/1412] 10.222.0.45 () {54 vars in 918 bytes} [Wed Aug 24 17:34:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:34:11.081848 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99ddea7-8a43-4a1d-a16b-cb421dcde831 None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:11.082139 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99ddea7-8a43-4a1d-a16b-cb421dcde831 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:11.082417 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99ddea7-8a43-4a1d-a16b-cb421dcde831 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:11.152886 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:34:11.157025 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-a99ddea7-8a43-4a1d-a16b-cb421dcde831 None None] MFA Rules not processed for user `053b7ce6d6f649ed91a26e434dab805e`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:34:11.158575 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-a99ddea7-8a43-4a1d-a16b-cb421dcde831 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:34:11.271903 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 706/1413] 10.222.0.45 () {58 vars in 1015 bytes} [Wed Aug 24 17:34:11 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:34:13.644340 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be95126-d1d2-4b8e-9960-64cac4374269 None None] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:13.644930 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be95126-d1d2-4b8e-9960-64cac4374269 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:13.644930 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be95126-d1d2-4b8e-9960-64cac4374269 None None] PATH_INFO: `/` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:13.646244 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 708/1414] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 17:34:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:34:13.651437 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626d98da-70ef-4a82-a5c6-cc09376405c7 None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:13.651900 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626d98da-70ef-4a82-a5c6-cc09376405c7 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:13.652008 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626d98da-70ef-4a82-a5c6-cc09376405c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:13.715744 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-626d98da-70ef-4a82-a5c6-cc09376405c7 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:34:13.716972 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-626d98da-70ef-4a82-a5c6-cc09376405c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:34:13.770840 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:34:13.813149 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 707/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 17:34:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:34:14.098652 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be95126-d1d2-4b8e-9960-64cac4374269 None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:14.100446 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-1be95126-d1d2-4b8e-9960-64cac4374269 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:34:14.242511 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be95126-d1d2-4b8e-9960-64cac4374269 None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:14.312943 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:34:14.352469 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35d81ec-16b2-4a55-b877-91fab15a96d1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '053b7ce6d6f649ed91a26e434dab805e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:14.353030 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35d81ec-16b2-4a55-b877-91fab15a96d1 service nova] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:14.353313 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35d81ec-16b2-4a55-b877-91fab15a96d1 service nova] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:14.353587 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35d81ec-16b2-4a55-b877-91fab15a96d1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:14.355169 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-c35d81ec-16b2-4a55-b877-91fab15a96d1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:34:14.441727 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35d81ec-16b2-4a55-b877-91fab15a96d1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:34:14.443332 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35d81ec-16b2-4a55-b877-91fab15a96d1 service nova] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:34:14.461311 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 709/1416] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:34:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:34:15.347125 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9ccf4b-5e48-4929-8a6a-b830dbbf2189 None None] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:15.347706 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9ccf4b-5e48-4929-8a6a-b830dbbf2189 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:15.347812 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9ccf4b-5e48-4929-8a6a-b830dbbf2189 None None] PATH_INFO: `/` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:15.349155 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 708/1417] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 17:34:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:34:15.354076 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0364050-ebff-4820-9f54-5c4a61ce1bc6 None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:15.354533 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0364050-ebff-4820-9f54-5c4a61ce1bc6 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:15.354650 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0364050-ebff-4820-9f54-5c4a61ce1bc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:15.428828 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-c0364050-ebff-4820-9f54-5c4a61ce1bc6 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:34:15.430019 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-c0364050-ebff-4820-9f54-5c4a61ce1bc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:34:15.456174 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:34:15.525634 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 710/1418] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 17:34:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:34:15.674627 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9ccf4b-5e48-4929-8a6a-b830dbbf2189 None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:15.676150 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-ed9ccf4b-5e48-4929-8a6a-b830dbbf2189 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:34:15.818627 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9ccf4b-5e48-4929-8a6a-b830dbbf2189 None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:15.857059 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:34:15.927716 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721c54b6-e2e0-4f65-af25-672a8b449c04 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '053b7ce6d6f649ed91a26e434dab805e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:15.928240 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721c54b6-e2e0-4f65-af25-672a8b449c04 service nova] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:15.928515 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721c54b6-e2e0-4f65-af25-672a8b449c04 service nova] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:15.928817 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721c54b6-e2e0-4f65-af25-672a8b449c04 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:15.930387 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-721c54b6-e2e0-4f65-af25-672a8b449c04 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:34:16.021100 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721c54b6-e2e0-4f65-af25-672a8b449c04 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:34:16.022674 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721c54b6-e2e0-4f65-af25-672a8b449c04 service nova] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:34:16.040111 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 709/1419] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:34:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:34:16.919293 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aeab690-efe2-46ec-97e6-cb889df33990 None None] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:16.919854 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aeab690-efe2-46ec-97e6-cb889df33990 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:16.919854 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aeab690-efe2-46ec-97e6-cb889df33990 None None] PATH_INFO: `/` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:16.921241 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 711/1420] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 17:34:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:34:16.926241 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266a5c72-c171-43cf-8f78-eb634101dbcc None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:16.926662 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266a5c72-c171-43cf-8f78-eb634101dbcc None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:16.926812 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266a5c72-c171-43cf-8f78-eb634101dbcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:16.980456 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:34:17.001511 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-266a5c72-c171-43cf-8f78-eb634101dbcc None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:34:17.002704 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-266a5c72-c171-43cf-8f78-eb634101dbcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:34:17.097100 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 710/1421] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 17:34:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:34:17.250146 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aeab690-efe2-46ec-97e6-cb889df33990 None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:17.251557 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-2aeab690-efe2-46ec-97e6-cb889df33990 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:34:17.392459 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aeab690-efe2-46ec-97e6-cb889df33990 None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:17.404156 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:34:17.502452 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f57cbeb-552e-4a8a-ada7-3a555f0c2218 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '053b7ce6d6f649ed91a26e434dab805e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:17.502980 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f57cbeb-552e-4a8a-ada7-3a555f0c2218 service nova] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:17.503259 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f57cbeb-552e-4a8a-ada7-3a555f0c2218 service nova] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:17.503533 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f57cbeb-552e-4a8a-ada7-3a555f0c2218 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:17.505137 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-2f57cbeb-552e-4a8a-ada7-3a555f0c2218 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:34:17.591653 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f57cbeb-552e-4a8a-ada7-3a555f0c2218 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:34:17.593349 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f57cbeb-552e-4a8a-ada7-3a555f0c2218 service nova] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:34:17.611212 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 712/1422] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:34:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:34:18.462171 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d024116c-3c00-4256-9150-6dd71b1c48f3 None None] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:18.462741 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d024116c-3c00-4256-9150-6dd71b1c48f3 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:18.462741 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d024116c-3c00-4256-9150-6dd71b1c48f3 None None] PATH_INFO: `/` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:18.464049 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 711/1423] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 17:34:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:34:18.469679 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73827678-055c-4846-bf78-c3ea6278cd46 None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:18.470111 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73827678-055c-4846-bf78-c3ea6278cd46 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:18.470226 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73827678-055c-4846-bf78-c3ea6278cd46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:18.489703 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:34:18.545798 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-73827678-055c-4846-bf78-c3ea6278cd46 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:34:18.547012 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-73827678-055c-4846-bf78-c3ea6278cd46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:34:18.644331 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 713/1424] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 17:34:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:34:18.793426 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d024116c-3c00-4256-9150-6dd71b1c48f3 None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:18.794985 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-d024116c-3c00-4256-9150-6dd71b1c48f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:34:18.903697 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:34:18.939233 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d024116c-3c00-4256-9150-6dd71b1c48f3 None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:19.050172 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8e9712-042e-4007-ba1d-e60ea4d01bb4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '053b7ce6d6f649ed91a26e434dab805e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:19.050704 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8e9712-042e-4007-ba1d-e60ea4d01bb4 service nova] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:19.050983 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8e9712-042e-4007-ba1d-e60ea4d01bb4 service nova] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:19.051256 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8e9712-042e-4007-ba1d-e60ea4d01bb4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:19.052876 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-8c8e9712-042e-4007-ba1d-e60ea4d01bb4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:34:19.134145 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:34:19.147841 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8e9712-042e-4007-ba1d-e60ea4d01bb4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:34:19.149476 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8e9712-042e-4007-ba1d-e60ea4d01bb4 service nova] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:34:19.167226 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 712/1425] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:34:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:34:20.033427 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1986ba1-9076-43cc-9039-4f269f985123 None None] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:20.034031 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1986ba1-9076-43cc-9039-4f269f985123 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:20.034031 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1986ba1-9076-43cc-9039-4f269f985123 None None] PATH_INFO: `/` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:20.035294 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 714/1426] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 17:34:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:34:20.040504 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b9ba01-b03c-474c-bcca-8c62255bb6f0 None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:20.040945 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b9ba01-b03c-474c-bcca-8c62255bb6f0 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:20.041083 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b9ba01-b03c-474c-bcca-8c62255bb6f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:20.117573 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-09b9ba01-b03c-474c-bcca-8c62255bb6f0 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:34:20.118689 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-09b9ba01-b03c-474c-bcca-8c62255bb6f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:34:20.213726 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 713/1427] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 17:34:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:34:20.364029 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1986ba1-9076-43cc-9039-4f269f985123 None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:20.365545 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-d1986ba1-9076-43cc-9039-4f269f985123 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:34:20.448723 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:34:20.510205 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1986ba1-9076-43cc-9039-4f269f985123 None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:20.618965 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdffcf7-c446-46d2-994c-32d1c90fa93b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '053b7ce6d6f649ed91a26e434dab805e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:20.619569 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdffcf7-c446-46d2-994c-32d1c90fa93b service nova] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:20.619849 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdffcf7-c446-46d2-994c-32d1c90fa93b service nova] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:20.620126 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdffcf7-c446-46d2-994c-32d1c90fa93b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:20.621986 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-ffdffcf7-c446-46d2-994c-32d1c90fa93b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:34:20.668615 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:34:20.708839 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdffcf7-c446-46d2-994c-32d1c90fa93b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:34:20.710372 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdffcf7-c446-46d2-994c-32d1c90fa93b service nova] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:34:20.728274 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 715/1428] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:34:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:34:21.528642 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d315909-b127-4e7f-bdf8-4ef7c95ab9a9 None None] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:21.529316 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d315909-b127-4e7f-bdf8-4ef7c95ab9a9 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:21.529316 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d315909-b127-4e7f-bdf8-4ef7c95ab9a9 None None] PATH_INFO: `/` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:21.530603 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 714/1429] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 17:34:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:34:21.535686 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9249b5-c52a-4262-8e58-ee99d76607ed None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:21.536113 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9249b5-c52a-4262-8e58-ee99d76607ed None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:21.536241 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9249b5-c52a-4262-8e58-ee99d76607ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:21.611686 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-0e9249b5-c52a-4262-8e58-ee99d76607ed None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:34:21.612916 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-0e9249b5-c52a-4262-8e58-ee99d76607ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:34:21.707779 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 716/1430] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 17:34:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:34:21.859701 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d315909-b127-4e7f-bdf8-4ef7c95ab9a9 None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:21.861308 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-5d315909-b127-4e7f-bdf8-4ef7c95ab9a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:34:21.908974 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:34:22.004309 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d315909-b127-4e7f-bdf8-4ef7c95ab9a9 None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:22.113423 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8d3523-061e-454e-bec9-4c1074617ac0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '053b7ce6d6f649ed91a26e434dab805e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:22.113956 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8d3523-061e-454e-bec9-4c1074617ac0 service nova] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:22.114232 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8d3523-061e-454e-bec9-4c1074617ac0 service nova] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:22.114503 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8d3523-061e-454e-bec9-4c1074617ac0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:22.116099 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-0f8d3523-061e-454e-bec9-4c1074617ac0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:34:22.123108 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:34:22.205043 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8d3523-061e-454e-bec9-4c1074617ac0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:34:22.206641 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8d3523-061e-454e-bec9-4c1074617ac0 service nova] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:34:22.224519 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 715/1431] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:34:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:34:23.020047 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077e3753-d6b6-49db-b6a7-b1e259342de2 None None] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:23.020630 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077e3753-d6b6-49db-b6a7-b1e259342de2 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:23.020630 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077e3753-d6b6-49db-b6a7-b1e259342de2 None None] PATH_INFO: `/` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:23.022030 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 717/1432] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 17:34:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:34:23.026795 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5481809a-0369-4b49-a238-6257984c68a5 None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:23.027216 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5481809a-0369-4b49-a238-6257984c68a5 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:23.027367 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5481809a-0369-4b49-a238-6257984c68a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:23.100965 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-5481809a-0369-4b49-a238-6257984c68a5 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:34:23.102175 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-5481809a-0369-4b49-a238-6257984c68a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:34:23.168763 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:34:23.198026 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 716/1433] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 17:34:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:34:23.347781 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077e3753-d6b6-49db-b6a7-b1e259342de2 None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:23.349269 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-077e3753-d6b6-49db-b6a7-b1e259342de2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:34:23.367042 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:34:23.493040 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077e3753-d6b6-49db-b6a7-b1e259342de2 None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:23.583328 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:34:23.611419 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704be273-bdc3-46fd-852c-46e8a2280d60 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '053b7ce6d6f649ed91a26e434dab805e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:23.611958 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704be273-bdc3-46fd-852c-46e8a2280d60 service nova] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:23.612244 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704be273-bdc3-46fd-852c-46e8a2280d60 service nova] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:23.612521 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704be273-bdc3-46fd-852c-46e8a2280d60 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:23.614145 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-704be273-bdc3-46fd-852c-46e8a2280d60 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:34:23.704114 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704be273-bdc3-46fd-852c-46e8a2280d60 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:34:23.705898 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704be273-bdc3-46fd-852c-46e8a2280d60 service nova] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:34:23.724731 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 718/1434] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:34:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:34:24.530866 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede3eccf-31e5-445a-ab3c-05704dd6c89c None None] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:24.531440 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede3eccf-31e5-445a-ab3c-05704dd6c89c None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:24.531440 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede3eccf-31e5-445a-ab3c-05704dd6c89c None None] PATH_INFO: `/` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:24.532780 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 717/1435] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 17:34:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:34:24.537812 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61a5686-fcc3-4075-98ef-d2d719c6e49f None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:24.538215 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61a5686-fcc3-4075-98ef-d2d719c6e49f None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:24.538371 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61a5686-fcc3-4075-98ef-d2d719c6e49f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:24.612222 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-c61a5686-fcc3-4075-98ef-d2d719c6e49f None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:34:24.613464 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-c61a5686-fcc3-4075-98ef-d2d719c6e49f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:34:24.654414 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:34:24.709902 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 719/1436] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 17:34:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:34:24.860877 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede3eccf-31e5-445a-ab3c-05704dd6c89c None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:24.862387 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-ede3eccf-31e5-445a-ab3c-05704dd6c89c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:34:25.004653 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede3eccf-31e5-445a-ab3c-05704dd6c89c None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:25.057380 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:34:25.115600 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c5ab6e-5ea2-4f94-a3c8-e89119be08ed service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '053b7ce6d6f649ed91a26e434dab805e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:25.116203 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c5ab6e-5ea2-4f94-a3c8-e89119be08ed service nova] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:25.116482 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c5ab6e-5ea2-4f94-a3c8-e89119be08ed service nova] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:25.116785 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c5ab6e-5ea2-4f94-a3c8-e89119be08ed service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:25.118377 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-80c5ab6e-5ea2-4f94-a3c8-e89119be08ed service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:34:25.215235 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c5ab6e-5ea2-4f94-a3c8-e89119be08ed service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:34:25.216844 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c5ab6e-5ea2-4f94-a3c8-e89119be08ed service nova] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:34:25.235058 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 718/1437] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:34:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:34:26.035214 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f036f712-7a40-4be3-a6ef-c8acdd77fd91 None None] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:26.035716 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f036f712-7a40-4be3-a6ef-c8acdd77fd91 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:26.035716 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f036f712-7a40-4be3-a6ef-c8acdd77fd91 None None] PATH_INFO: `/` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:26.037078 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 720/1438] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 17:34:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:34:26.041987 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e23319-484d-449d-8abc-cb7247fe642a None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:26.042446 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e23319-484d-449d-8abc-cb7247fe642a None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:26.042556 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e23319-484d-449d-8abc-cb7247fe642a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:26.117767 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-d1e23319-484d-449d-8abc-cb7247fe642a None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:34:26.118953 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-d1e23319-484d-449d-8abc-cb7247fe642a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:34:26.121211 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:34:26.214880 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 719/1439] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 17:34:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:34:26.366037 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f036f712-7a40-4be3-a6ef-c8acdd77fd91 None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:26.367636 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-f036f712-7a40-4be3-a6ef-c8acdd77fd91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:34:26.516452 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f036f712-7a40-4be3-a6ef-c8acdd77fd91 None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:26.543022 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:34:26.629620 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7931342f-9fd3-4090-ac06-0f4788166946 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '053b7ce6d6f649ed91a26e434dab805e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:26.630148 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7931342f-9fd3-4090-ac06-0f4788166946 service nova] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:26.630426 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7931342f-9fd3-4090-ac06-0f4788166946 service nova] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:26.630699 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7931342f-9fd3-4090-ac06-0f4788166946 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:26.632277 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-7931342f-9fd3-4090-ac06-0f4788166946 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:34:26.718821 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7931342f-9fd3-4090-ac06-0f4788166946 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:34:26.720419 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7931342f-9fd3-4090-ac06-0f4788166946 service nova] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:34:26.738109 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 721/1440] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:34:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:34:28.557843 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57aabd23-6754-44e2-b65b-cfd468ca8394 None None] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:28.558445 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57aabd23-6754-44e2-b65b-cfd468ca8394 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:28.558445 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57aabd23-6754-44e2-b65b-cfd468ca8394 None None] PATH_INFO: `/` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:28.559741 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 720/1441] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 17:34:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:34:28.565900 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389bfc63-cbc5-4d34-b5a4-789a0655e0e1 None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:28.566345 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389bfc63-cbc5-4d34-b5a4-789a0655e0e1 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:28.566453 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389bfc63-cbc5-4d34-b5a4-789a0655e0e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:28.606376 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:34:28.631392 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-389bfc63-cbc5-4d34-b5a4-789a0655e0e1 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:34:28.632611 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-389bfc63-cbc5-4d34-b5a4-789a0655e0e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:34:28.726636 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 722/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 17:34:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:34:29.009923 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57aabd23-6754-44e2-b65b-cfd468ca8394 None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:29.011590 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-57aabd23-6754-44e2-b65b-cfd468ca8394 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:34:29.117150 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57aabd23-6754-44e2-b65b-cfd468ca8394 None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:29.149498 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:34:29.191277 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2e7dd2-0ce6-4e3d-ae04-1fb9f9260e09 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43ad98a250a144099ccd18378aedd4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:29.191881 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2e7dd2-0ce6-4e3d-ae04-1fb9f9260e09 service neutron] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:29.192156 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2e7dd2-0ce6-4e3d-ae04-1fb9f9260e09 service neutron] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:29.192428 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2e7dd2-0ce6-4e3d-ae04-1fb9f9260e09 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:29.194515 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-dd2e7dd2-0ce6-4e3d-ae04-1fb9f9260e09 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:34:29.301265 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2e7dd2-0ce6-4e3d-ae04-1fb9f9260e09 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:34:29.302723 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2e7dd2-0ce6-4e3d-ae04-1fb9f9260e09 service neutron] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:34:29.322118 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 721/1443] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:34:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:34:31.407580 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b947400b-9d2d-4d26-81c0-bfc937a5c18a None None] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:31.408155 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b947400b-9d2d-4d26-81c0-bfc937a5c18a None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:31.408155 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b947400b-9d2d-4d26-81c0-bfc937a5c18a None None] PATH_INFO: `/` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:31.409496 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 723/1444] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 17:34:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:34:31.414957 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39733c3c-7334-400c-be72-45e429f3e88f None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:31.415394 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39733c3c-7334-400c-be72-45e429f3e88f None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:31.415508 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39733c3c-7334-400c-be72-45e429f3e88f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:31.479435 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-39733c3c-7334-400c-be72-45e429f3e88f None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:34:31.480676 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-39733c3c-7334-400c-be72-45e429f3e88f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:34:31.493569 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:34:31.577089 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 722/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 17:34:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:34:31.859150 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b947400b-9d2d-4d26-81c0-bfc937a5c18a None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:31.860736 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-b947400b-9d2d-4d26-81c0-bfc937a5c18a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:34:31.959997 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b947400b-9d2d-4d26-81c0-bfc937a5c18a None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:31.972225 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:34:32.032543 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bb3fb6-567f-4ed8-a399-a6812a067b58 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43ad98a250a144099ccd18378aedd4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:32.033170 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bb3fb6-567f-4ed8-a399-a6812a067b58 service neutron] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:32.033448 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bb3fb6-567f-4ed8-a399-a6812a067b58 service neutron] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:32.033719 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bb3fb6-567f-4ed8-a399-a6812a067b58 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:32.035293 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-31bb3fb6-567f-4ed8-a399-a6812a067b58 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:34:32.140033 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bb3fb6-567f-4ed8-a399-a6812a067b58 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:34:32.141409 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bb3fb6-567f-4ed8-a399-a6812a067b58 service neutron] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:34:32.159406 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 724/1446] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:34:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:34:34.432127 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cca305-a9a7-4ad3-be2d-84cf63f1c1a4 None None] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:34.432711 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cca305-a9a7-4ad3-be2d-84cf63f1c1a4 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:34.432711 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cca305-a9a7-4ad3-be2d-84cf63f1c1a4 None None] PATH_INFO: `/` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:34.434088 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 723/1447] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 17:34:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:34:34.439788 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6028b69a-8309-4f97-8496-60f7c2597113 None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:34.440210 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6028b69a-8309-4f97-8496-60f7c2597113 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:34.440339 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6028b69a-8309-4f97-8496-60f7c2597113 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:34.487439 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:34:34.505509 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-6028b69a-8309-4f97-8496-60f7c2597113 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:34:34.506717 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-6028b69a-8309-4f97-8496-60f7c2597113 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:34:34.600783 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 725/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 17:34:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:34:34.893355 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cca305-a9a7-4ad3-be2d-84cf63f1c1a4 None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:34.894923 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-22cca305-a9a7-4ad3-be2d-84cf63f1c1a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:34:34.996433 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cca305-a9a7-4ad3-be2d-84cf63f1c1a4 None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:35.036022 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:34:35.069183 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431723fe-6605-4e5d-8c35-a9b05279ff75 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43ad98a250a144099ccd18378aedd4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:35.069791 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431723fe-6605-4e5d-8c35-a9b05279ff75 service neutron] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:35.070073 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431723fe-6605-4e5d-8c35-a9b05279ff75 service neutron] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:35.070347 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431723fe-6605-4e5d-8c35-a9b05279ff75 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:35.071970 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-431723fe-6605-4e5d-8c35-a9b05279ff75 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:34:35.178632 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431723fe-6605-4e5d-8c35-a9b05279ff75 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:34:35.180054 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431723fe-6605-4e5d-8c35-a9b05279ff75 service neutron] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:34:35.198389 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 724/1449] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:34:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:34:37.209131 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ae0f62-694b-40bf-a2cb-d942e5f484d1 None None] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:37.209692 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ae0f62-694b-40bf-a2cb-d942e5f484d1 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:37.209692 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ae0f62-694b-40bf-a2cb-d942e5f484d1 None None] PATH_INFO: `/` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:37.211006 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 726/1450] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 17:34:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:34:37.216230 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71af6c08-9528-41f7-bacc-3b6f16995636 None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:37.216648 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71af6c08-9528-41f7-bacc-3b6f16995636 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:37.216822 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71af6c08-9528-41f7-bacc-3b6f16995636 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:37.282940 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-71af6c08-9528-41f7-bacc-3b6f16995636 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:34:37.284243 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-71af6c08-9528-41f7-bacc-3b6f16995636 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:34:37.304259 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:34:37.378937 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 725/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 17:34:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:34:37.661704 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ae0f62-694b-40bf-a2cb-d942e5f484d1 None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:37.663219 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-76ae0f62-694b-40bf-a2cb-d942e5f484d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:34:37.764413 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ae0f62-694b-40bf-a2cb-d942e5f484d1 None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:37.784641 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:34:37.838087 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64294731-c8c4-4a1e-82fd-cbc48d8a75ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43ad98a250a144099ccd18378aedd4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:37.838687 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64294731-c8c4-4a1e-82fd-cbc48d8a75ce service neutron] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:37.838963 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64294731-c8c4-4a1e-82fd-cbc48d8a75ce service neutron] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:37.839232 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64294731-c8c4-4a1e-82fd-cbc48d8a75ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:37.840812 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-64294731-c8c4-4a1e-82fd-cbc48d8a75ce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:34:37.946978 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64294731-c8c4-4a1e-82fd-cbc48d8a75ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:34:37.948343 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64294731-c8c4-4a1e-82fd-cbc48d8a75ce service neutron] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:34:37.966204 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 727/1452] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:34:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:34:39.921018 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45446802-a5c9-4d89-8384-e937dd1a3d42 None None] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:39.921583 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45446802-a5c9-4d89-8384-e937dd1a3d42 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:39.921583 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45446802-a5c9-4d89-8384-e937dd1a3d42 None None] PATH_INFO: `/` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:39.922928 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 726/1453] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 17:34:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:34:39.928134 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295ddac9-6777-4ac0-bd66-2e4d39916aa9 None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:39.928524 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295ddac9-6777-4ac0-bd66-2e4d39916aa9 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:39.928733 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295ddac9-6777-4ac0-bd66-2e4d39916aa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:39.991077 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:34:39.993757 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-295ddac9-6777-4ac0-bd66-2e4d39916aa9 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:34:39.995639 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-295ddac9-6777-4ac0-bd66-2e4d39916aa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:34:40.090459 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 728/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 17:34:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:34:40.373064 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45446802-a5c9-4d89-8384-e937dd1a3d42 None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:40.375233 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-45446802-a5c9-4d89-8384-e937dd1a3d42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:34:40.476255 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45446802-a5c9-4d89-8384-e937dd1a3d42 None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:40.525462 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:34:40.550667 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b11c37a-a7ce-41b3-a8a5-30b106c6b4bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43ad98a250a144099ccd18378aedd4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:40.551262 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b11c37a-a7ce-41b3-a8a5-30b106c6b4bb service neutron] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:40.551537 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b11c37a-a7ce-41b3-a8a5-30b106c6b4bb service neutron] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:40.551801 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b11c37a-a7ce-41b3-a8a5-30b106c6b4bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:40.553393 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-8b11c37a-a7ce-41b3-a8a5-30b106c6b4bb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:34:40.658503 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b11c37a-a7ce-41b3-a8a5-30b106c6b4bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:34:40.659934 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b11c37a-a7ce-41b3-a8a5-30b106c6b4bb service neutron] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:34:40.678038 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 727/1455] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:34:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:34:42.629288 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69c420e-6487-4952-bab3-0f87c0bb2689 None None] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:42.629854 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69c420e-6487-4952-bab3-0f87c0bb2689 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:42.629854 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69c420e-6487-4952-bab3-0f87c0bb2689 None None] PATH_INFO: `/` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:42.631560 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 729/1456] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 17:34:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 17:34:42.637106 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e3c4b5-f810-4269-9370-6dd07a3a28b8 None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:42.637534 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e3c4b5-f810-4269-9370-6dd07a3a28b8 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:42.637640 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e3c4b5-f810-4269-9370-6dd07a3a28b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:42.703449 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-43e3c4b5-f810-4269-9370-6dd07a3a28b8 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:34:42.704657 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-43e3c4b5-f810-4269-9370-6dd07a3a28b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:34:42.732704 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:34:42.801133 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 728/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 17:34:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:34:43.085528 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69c420e-6487-4952-bab3-0f87c0bb2689 None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:43.087047 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-f69c420e-6487-4952-bab3-0f87c0bb2689 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:34:43.192357 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69c420e-6487-4952-bab3-0f87c0bb2689 None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:43.221724 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:34:43.266930 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506f91ad-a795-4cb2-9e8e-33783e323cc7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43ad98a250a144099ccd18378aedd4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:43.267533 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506f91ad-a795-4cb2-9e8e-33783e323cc7 service neutron] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:43.267812 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506f91ad-a795-4cb2-9e8e-33783e323cc7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:43.268082 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506f91ad-a795-4cb2-9e8e-33783e323cc7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:43.269760 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-506f91ad-a795-4cb2-9e8e-33783e323cc7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:34:43.377438 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506f91ad-a795-4cb2-9e8e-33783e323cc7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:34:43.378790 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506f91ad-a795-4cb2-9e8e-33783e323cc7 service neutron] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:34:43.396234 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 730/1458] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:34:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:34:45.332932 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9c98d2-5ce2-4b5f-8ae1-8ef01a4d4b64 None None] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:45.333545 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9c98d2-5ce2-4b5f-8ae1-8ef01a4d4b64 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:45.333545 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9c98d2-5ce2-4b5f-8ae1-8ef01a4d4b64 None None] PATH_INFO: `/` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:45.334795 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 729/1459] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 17:34:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:34:45.340000 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63d93f3-7fba-412a-9941-a1cdd13a9cf2 None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:45.340429 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63d93f3-7fba-412a-9941-a1cdd13a9cf2 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:45.340582 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63d93f3-7fba-412a-9941-a1cdd13a9cf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:45.406374 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-b63d93f3-7fba-412a-9941-a1cdd13a9cf2 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:34:45.407582 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-b63d93f3-7fba-412a-9941-a1cdd13a9cf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:34:45.415817 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:34:45.500837 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 731/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 17:34:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:34:45.786435 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9c98d2-5ce2-4b5f-8ae1-8ef01a4d4b64 None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:45.787968 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-3b9c98d2-5ce2-4b5f-8ae1-8ef01a4d4b64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:34:45.890614 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9c98d2-5ce2-4b5f-8ae1-8ef01a4d4b64 None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:45.943921 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:34:45.962503 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c0171f-412d-42ca-9de5-7449ed9496a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43ad98a250a144099ccd18378aedd4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:45.963110 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c0171f-412d-42ca-9de5-7449ed9496a1 service neutron] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:45.963392 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c0171f-412d-42ca-9de5-7449ed9496a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:45.963671 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c0171f-412d-42ca-9de5-7449ed9496a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:45.965308 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-20c0171f-412d-42ca-9de5-7449ed9496a1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:34:46.071586 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c0171f-412d-42ca-9de5-7449ed9496a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:34:46.072994 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c0171f-412d-42ca-9de5-7449ed9496a1 service neutron] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:34:46.091314 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 730/1461] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:34:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:34:48.020724 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f55146e-2847-4e9d-aa36-ab4a5cd7ac9c None None] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:48.021280 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f55146e-2847-4e9d-aa36-ab4a5cd7ac9c None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:48.021280 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f55146e-2847-4e9d-aa36-ab4a5cd7ac9c None None] PATH_INFO: `/` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:48.022666 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 732/1462] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 17:34:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:34:48.028066 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c984a33c-d200-4dfe-86ea-d948d9c34ae3 None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:48.028489 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c984a33c-d200-4dfe-86ea-d948d9c34ae3 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:48.028627 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c984a33c-d200-4dfe-86ea-d948d9c34ae3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:48.094180 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-c984a33c-d200-4dfe-86ea-d948d9c34ae3 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:34:48.095306 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-c984a33c-d200-4dfe-86ea-d948d9c34ae3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:34:48.133536 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:34:48.192217 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 731/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 17:34:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:34:48.472282 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f55146e-2847-4e9d-aa36-ab4a5cd7ac9c None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:48.474566 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-6f55146e-2847-4e9d-aa36-ab4a5cd7ac9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:34:48.576387 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f55146e-2847-4e9d-aa36-ab4a5cd7ac9c None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:48.612443 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:34:48.649546 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00488308-661c-4905-b50e-aa304147430b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43ad98a250a144099ccd18378aedd4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:48.650148 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00488308-661c-4905-b50e-aa304147430b service neutron] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:48.650422 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00488308-661c-4905-b50e-aa304147430b service neutron] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:48.650690 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00488308-661c-4905-b50e-aa304147430b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:48.652307 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-00488308-661c-4905-b50e-aa304147430b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:34:48.760962 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00488308-661c-4905-b50e-aa304147430b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:34:48.762451 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00488308-661c-4905-b50e-aa304147430b service neutron] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:34:48.780609 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 733/1464] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:34:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:34:50.710874 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c88d128-38a7-4f3b-9dc8-8062defbe830 None None] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:50.711424 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c88d128-38a7-4f3b-9dc8-8062defbe830 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:50.711424 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c88d128-38a7-4f3b-9dc8-8062defbe830 None None] PATH_INFO: `/` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:50.712803 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 732/1465] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 17:34:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:34:50.718846 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d6fbff-d7b3-4c1a-bb04-d07fa28ca54f None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:50.719267 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d6fbff-d7b3-4c1a-bb04-d07fa28ca54f None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:50.719372 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d6fbff-d7b3-4c1a-bb04-d07fa28ca54f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:50.786743 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-30d6fbff-d7b3-4c1a-bb04-d07fa28ca54f None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:34:50.787967 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-30d6fbff-d7b3-4c1a-bb04-d07fa28ca54f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:34:50.805917 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:34:50.881911 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 734/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 17:34:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:34:51.166473 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c88d128-38a7-4f3b-9dc8-8062defbe830 None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:51.168027 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-3c88d128-38a7-4f3b-9dc8-8062defbe830 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:34:51.272786 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c88d128-38a7-4f3b-9dc8-8062defbe830 None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:51.345127 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:34:51.352208 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0647166-71cc-4a32-9b4d-deb9c1326864 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43ad98a250a144099ccd18378aedd4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:51.352834 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0647166-71cc-4a32-9b4d-deb9c1326864 service neutron] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:51.353108 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0647166-71cc-4a32-9b4d-deb9c1326864 service neutron] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:51.353374 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0647166-71cc-4a32-9b4d-deb9c1326864 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:51.354952 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-e0647166-71cc-4a32-9b4d-deb9c1326864 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:34:51.460348 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0647166-71cc-4a32-9b4d-deb9c1326864 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:34:51.461719 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0647166-71cc-4a32-9b4d-deb9c1326864 service neutron] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:34:51.479316 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 733/1467] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:34:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:34:53.380022 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d85bd0-1b0c-4979-a1b1-3fd22c350b1d None None] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:53.380588 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d85bd0-1b0c-4979-a1b1-3fd22c350b1d None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:53.380588 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d85bd0-1b0c-4979-a1b1-3fd22c350b1d None None] PATH_INFO: `/` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:53.381924 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 735/1468] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 17:34:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:34:53.387252 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970d048b-3e6a-4b29-965f-6a631d56d0f5 None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:53.387663 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970d048b-3e6a-4b29-965f-6a631d56d0f5 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:53.387805 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970d048b-3e6a-4b29-965f-6a631d56d0f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:53.452871 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-970d048b-3e6a-4b29-965f-6a631d56d0f5 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:34:53.454058 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-970d048b-3e6a-4b29-965f-6a631d56d0f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:34:53.501092 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:34:53.549141 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 734/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 17:34:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:34:53.835769 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d85bd0-1b0c-4979-a1b1-3fd22c350b1d None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:53.837327 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-e9d85bd0-1b0c-4979-a1b1-3fd22c350b1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:34:53.942567 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d85bd0-1b0c-4979-a1b1-3fd22c350b1d None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:53.988292 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:34:54.014553 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6328a1-dfdf-4aaa-8d74-0830e4424df9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43ad98a250a144099ccd18378aedd4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:54.015181 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6328a1-dfdf-4aaa-8d74-0830e4424df9 service neutron] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:54.015460 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6328a1-dfdf-4aaa-8d74-0830e4424df9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:54.015734 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6328a1-dfdf-4aaa-8d74-0830e4424df9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:54.017347 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-eb6328a1-dfdf-4aaa-8d74-0830e4424df9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:34:54.126743 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6328a1-dfdf-4aaa-8d74-0830e4424df9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:34:54.128202 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6328a1-dfdf-4aaa-8d74-0830e4424df9 service neutron] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:34:54.145985 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 736/1470] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:34:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:35:03.547340 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=22 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:04.143738 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=66 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:13.532985 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b6ae8d-fe35-43a8-9eb1-07ccdc85e0d7 None None] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:13.533553 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b6ae8d-fe35-43a8-9eb1-07ccdc85e0d7 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:13.533553 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b6ae8d-fe35-43a8-9eb1-07ccdc85e0d7 None None] PATH_INFO: `/` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:13.534924 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 735/1471] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:35:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:35:13.540210 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4c6f44-3af6-4110-9d56-0fd9adb2e1df None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:13.540612 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4c6f44-3af6-4110-9d56-0fd9adb2e1df None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:13.540813 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4c6f44-3af6-4110-9d56-0fd9adb2e1df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:13.604386 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-6e4c6f44-3af6-4110-9d56-0fd9adb2e1df None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:35:13.605605 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-6e4c6f44-3af6-4110-9d56-0fd9adb2e1df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:35:13.817670 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 737/1472] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:35:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:35:14.115308 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b62123-a5f7-41de-894b-b26e6a1d663a None None] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:14.115592 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b62123-a5f7-41de-894b-b26e6a1d663a None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:14.115868 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b62123-a5f7-41de-894b-b26e6a1d663a None None] PATH_INFO: `/` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:14.117222 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 736/1473] 10.222.0.45 () {54 vars in 967 bytes} [Wed Aug 24 17:35:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:35:14.122048 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d304b6d-bc58-413a-9dcc-72b400c0c38e None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:14.122336 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d304b6d-bc58-413a-9dcc-72b400c0c38e None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:14.122863 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d304b6d-bc58-413a-9dcc-72b400c0c38e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:14.196609 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-4d304b6d-bc58-413a-9dcc-72b400c0c38e None None] MFA Rules not processed for user `df5fee57539447428dd1c72eab3cfd9d`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:35:14.197780 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-4d304b6d-bc58-413a-9dcc-72b400c0c38e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:35:14.217948 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:14.273289 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 738/1474] 10.222.0.45 () {58 vars in 1064 bytes} [Wed Aug 24 17:35:14 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:35:14.281440 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b62123-a5f7-41de-894b-b26e6a1d663a None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:14.282948 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-d6b62123-a5f7-41de-894b-b26e6a1d663a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:35:14.384694 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b62123-a5f7-41de-894b-b26e6a1d663a None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:14.455698 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e9c2f3-6e13-4f5c-84e5-14c89a884b6d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df5fee57539447428dd1c72eab3cfd9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:14.456297 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e9c2f3-6e13-4f5c-84e5-14c89a884b6d service glance] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:14.456608 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e9c2f3-6e13-4f5c-84e5-14c89a884b6d service glance] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:14.456889 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e9c2f3-6e13-4f5c-84e5-14c89a884b6d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:14.458474 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-b3e9c2f3-6e13-4f5c-84e5-14c89a884b6d service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:35:14.512482 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:14.566952 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e9c2f3-6e13-4f5c-84e5-14c89a884b6d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:14.568396 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e9c2f3-6e13-4f5c-84e5-14c89a884b6d service glance] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:14.586411 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 737/1475] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:35:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:35:16.551348 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7ed519-bca0-4e8f-a762-40927f812760 None None] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:16.551913 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7ed519-bca0-4e8f-a762-40927f812760 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:16.551913 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7ed519-bca0-4e8f-a762-40927f812760 None None] PATH_INFO: `/` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:16.553264 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 739/1476] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:35:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:35:16.558827 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61b6a8b-4e3a-4a85-9d30-84dad11b0fdd None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:16.559282 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61b6a8b-4e3a-4a85-9d30-84dad11b0fdd None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:16.559394 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61b6a8b-4e3a-4a85-9d30-84dad11b0fdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:16.624678 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-c61b6a8b-4e3a-4a85-9d30-84dad11b0fdd None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:35:16.625897 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-c61b6a8b-4e3a-4a85-9d30-84dad11b0fdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:35:16.720111 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 738/1477] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:35:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:35:16.988865 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7ed519-bca0-4e8f-a762-40927f812760 None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:16.990446 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-0b7ed519-bca0-4e8f-a762-40927f812760 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:35:17.111940 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7ed519-bca0-4e8f-a762-40927f812760 None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:17.159178 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:17.203834 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffcc671-ca4f-4833-b5d8-2a470f1356c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:17.204434 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffcc671-ca4f-4833-b5d8-2a470f1356c7 admin admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:17.204745 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffcc671-ca4f-4833-b5d8-2a470f1356c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:17.205020 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffcc671-ca4f-4833-b5d8-2a470f1356c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:17.205659 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffcc671-ca4f-4833-b5d8-2a470f1356c7 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:17.207383 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: WARNING py.warnings [None req-dffcc671-ca4f-4833-b5d8-2a470f1356c7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 17:35:17.207383 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: warnings.warn(msg) Aug 24 17:35:17.207383 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: Aug 24 17:35:17.208509 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffcc671-ca4f-4833-b5d8-2a470f1356c7 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:17.226737 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 740/1478] 10.222.0.45 () {56 vars in 1108 bytes} [Wed Aug 24 17:35:16 2022] GET /identity/v3/projects => generated 2553 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:35:19.124541 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f390b5d1-e83d-4c20-b92b-795055f3580a None None] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:19.125084 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f390b5d1-e83d-4c20-b92b-795055f3580a None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:19.125084 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f390b5d1-e83d-4c20-b92b-795055f3580a None None] PATH_INFO: `/` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:19.126353 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 739/1479] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:35:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:35:19.131524 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f9a035-3f04-48c2-abcb-71bf7b64a07f None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:19.131951 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f9a035-3f04-48c2-abcb-71bf7b64a07f None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:19.132086 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f9a035-3f04-48c2-abcb-71bf7b64a07f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:19.197133 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-f7f9a035-3f04-48c2-abcb-71bf7b64a07f None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:35:19.198513 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-f7f9a035-3f04-48c2-abcb-71bf7b64a07f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:35:19.295180 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 741/1480] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:35:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:35:19.588891 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f390b5d1-e83d-4c20-b92b-795055f3580a None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:19.590543 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-f390b5d1-e83d-4c20-b92b-795055f3580a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:35:19.600354 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:19.736605 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f390b5d1-e83d-4c20-b92b-795055f3580a None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:19.815246 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:19.849348 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b91a488-e4f8-47cf-b4eb-1a822f90102f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '053b7ce6d6f649ed91a26e434dab805e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:19.849885 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b91a488-e4f8-47cf-b4eb-1a822f90102f service nova] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:19.850158 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b91a488-e4f8-47cf-b4eb-1a822f90102f service nova] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:19.850428 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b91a488-e4f8-47cf-b4eb-1a822f90102f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:19.852027 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-1b91a488-e4f8-47cf-b4eb-1a822f90102f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:35:19.939205 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b91a488-e4f8-47cf-b4eb-1a822f90102f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:19.940838 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b91a488-e4f8-47cf-b4eb-1a822f90102f service nova] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:19.958273 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 740/1481] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:35:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:35:22.009090 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a27b76-9abf-4b85-b91d-6a6e3915635c None None] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:22.009646 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a27b76-9abf-4b85-b91d-6a6e3915635c None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:22.009646 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a27b76-9abf-4b85-b91d-6a6e3915635c None None] PATH_INFO: `/` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:22.010956 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 742/1482] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:35:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:35:22.016202 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9278b31c-e603-479c-83db-aefa3650070e None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:22.016651 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9278b31c-e603-479c-83db-aefa3650070e None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:22.016780 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9278b31c-e603-479c-83db-aefa3650070e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:22.080993 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-9278b31c-e603-479c-83db-aefa3650070e None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:35:22.082138 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-9278b31c-e603-479c-83db-aefa3650070e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:35:22.127988 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:22.178546 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 741/1483] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:35:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:35:22.466856 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a27b76-9abf-4b85-b91d-6a6e3915635c None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:22.469033 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-c8a27b76-9abf-4b85-b91d-6a6e3915635c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:35:22.492041 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:22.569982 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a27b76-9abf-4b85-b91d-6a6e3915635c None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:22.640122 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9039fd-5ca2-47b5-9935-3a0d84e7010a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43ad98a250a144099ccd18378aedd4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:22.640747 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9039fd-5ca2-47b5-9935-3a0d84e7010a service neutron] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:22.641032 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9039fd-5ca2-47b5-9935-3a0d84e7010a service neutron] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:22.641294 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9039fd-5ca2-47b5-9935-3a0d84e7010a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:22.642856 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-be9039fd-5ca2-47b5-9935-3a0d84e7010a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:35:22.713842 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:22.749089 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9039fd-5ca2-47b5-9935-3a0d84e7010a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:22.750505 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9039fd-5ca2-47b5-9935-3a0d84e7010a service neutron] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:22.768065 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 743/1484] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:35:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:35:24.703872 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f69c8c2-bdc1-427a-939b-f1b83b785fe6 None None] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:24.704390 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f69c8c2-bdc1-427a-939b-f1b83b785fe6 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:24.704511 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f69c8c2-bdc1-427a-939b-f1b83b785fe6 None None] PATH_INFO: `/` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:24.705839 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 742/1485] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:35:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:35:24.711191 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72f8dc5-0c19-4107-bb49-1140e884af23 None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:24.711671 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72f8dc5-0c19-4107-bb49-1140e884af23 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:24.711768 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72f8dc5-0c19-4107-bb49-1140e884af23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:24.780660 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-f72f8dc5-0c19-4107-bb49-1140e884af23 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:35:24.781906 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-f72f8dc5-0c19-4107-bb49-1140e884af23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:35:24.877909 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 744/1486] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:35:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:35:25.168484 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f69c8c2-bdc1-427a-939b-f1b83b785fe6 None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:25.170087 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-7f69c8c2-bdc1-427a-939b-f1b83b785fe6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:35:25.273305 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f69c8c2-bdc1-427a-939b-f1b83b785fe6 None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:25.344771 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:25.346507 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8547379-37d4-4531-8b4d-6c63e5ea8a91 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43ad98a250a144099ccd18378aedd4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:25.347119 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8547379-37d4-4531-8b4d-6c63e5ea8a91 service neutron] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:25.347578 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8547379-37d4-4531-8b4d-6c63e5ea8a91 service neutron] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:25.347886 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8547379-37d4-4531-8b4d-6c63e5ea8a91 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:25.349840 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-c8547379-37d4-4531-8b4d-6c63e5ea8a91 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:35:25.456839 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8547379-37d4-4531-8b4d-6c63e5ea8a91 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:25.458202 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8547379-37d4-4531-8b4d-6c63e5ea8a91 service neutron] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:25.476179 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 743/1487] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:35:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:35:27.528248 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f39b5d-9e48-4488-a3f9-2a791c357c9c None None] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:27.528863 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f39b5d-9e48-4488-a3f9-2a791c357c9c None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:27.528863 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f39b5d-9e48-4488-a3f9-2a791c357c9c None None] PATH_INFO: `/` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:27.530232 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 745/1488] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:35:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:35:27.535416 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f968c69-5d90-47ff-ab6b-144bca80054b None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:27.535799 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f968c69-5d90-47ff-ab6b-144bca80054b None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:27.535964 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f968c69-5d90-47ff-ab6b-144bca80054b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:27.599743 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-2f968c69-5d90-47ff-ab6b-144bca80054b None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:35:27.600983 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-2f968c69-5d90-47ff-ab6b-144bca80054b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:35:27.653110 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:27.696875 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 744/1489] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:35:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:35:27.990844 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f39b5d-9e48-4488-a3f9-2a791c357c9c None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:27.992366 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-e1f39b5d-9e48-4488-a3f9-2a791c357c9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:35:28.027347 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:28.093811 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f39b5d-9e48-4488-a3f9-2a791c357c9c None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:28.165256 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d283c38-aa50-4e97-ae5b-d8406cb1b3ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43ad98a250a144099ccd18378aedd4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:28.165864 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d283c38-aa50-4e97-ae5b-d8406cb1b3ad service neutron] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:28.166144 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d283c38-aa50-4e97-ae5b-d8406cb1b3ad service neutron] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:28.166423 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d283c38-aa50-4e97-ae5b-d8406cb1b3ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:28.167996 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-0d283c38-aa50-4e97-ae5b-d8406cb1b3ad service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:35:28.248911 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:28.277548 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d283c38-aa50-4e97-ae5b-d8406cb1b3ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:28.278988 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d283c38-aa50-4e97-ae5b-d8406cb1b3ad service neutron] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:28.297016 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 746/1490] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:35:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:35:30.353327 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5af410e-db74-4784-93e9-f5da4fb40162 None None] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:30.353975 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5af410e-db74-4784-93e9-f5da4fb40162 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:30.353975 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5af410e-db74-4784-93e9-f5da4fb40162 None None] PATH_INFO: `/` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:30.355217 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 745/1491] 10.222.0.45 () {54 vars in 927 bytes} [Wed Aug 24 17:35:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:35:30.361312 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3196ef-15c4-4ed7-be1f-55ac3f7605c3 None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:30.361746 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3196ef-15c4-4ed7-be1f-55ac3f7605c3 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:30.361874 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3196ef-15c4-4ed7-be1f-55ac3f7605c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:30.427113 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-bd3196ef-15c4-4ed7-be1f-55ac3f7605c3 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:35:30.428342 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-bd3196ef-15c4-4ed7-be1f-55ac3f7605c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:35:30.523037 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 747/1492] 10.222.0.45 () {58 vars in 1024 bytes} [Wed Aug 24 17:35:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:35:30.806763 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5af410e-db74-4784-93e9-f5da4fb40162 None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:30.808328 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-e5af410e-db74-4784-93e9-f5da4fb40162 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:35:30.909699 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5af410e-db74-4784-93e9-f5da4fb40162 None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:30.979916 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96f1246-24db-464f-8336-10815b8384ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43ad98a250a144099ccd18378aedd4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:30.980511 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96f1246-24db-464f-8336-10815b8384ef service neutron] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:30.980815 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96f1246-24db-464f-8336-10815b8384ef service neutron] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:30.981088 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96f1246-24db-464f-8336-10815b8384ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:30.982653 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-a96f1246-24db-464f-8336-10815b8384ef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:35:30.991580 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:31.091171 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96f1246-24db-464f-8336-10815b8384ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:31.092574 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96f1246-24db-464f-8336-10815b8384ef service neutron] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:31.111316 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 746/1493] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:35:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:35:40.521252 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=82 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:41.108434 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=50 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:48.432745 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183c2da9-5662-47b5-9afa-c8a309efb134 None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:48.433337 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183c2da9-5662-47b5-9afa-c8a309efb134 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:48.433337 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183c2da9-5662-47b5-9afa-c8a309efb134 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:48.508578 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-183c2da9-5662-47b5-9afa-c8a309efb134 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:36:48.509804 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-183c2da9-5662-47b5-9afa-c8a309efb134 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:36:48.602907 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 748/1494] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:36:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:36:48.610953 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22093bf-1495-4f9e-bb18-a50633408e37 None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:48.611465 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22093bf-1495-4f9e-bb18-a50633408e37 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:48.611564 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22093bf-1495-4f9e-bb18-a50633408e37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:48.686581 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-e22093bf-1495-4f9e-bb18-a50633408e37 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:36:48.687758 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-e22093bf-1495-4f9e-bb18-a50633408e37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:36:48.784391 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 747/1495] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:36:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:36:48.789050 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183c2da9-5662-47b5-9afa-c8a309efb134 None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:48.790246 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-183c2da9-5662-47b5-9afa-c8a309efb134 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:36:48.802017 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183c2da9-5662-47b5-9afa-c8a309efb134 None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:48.862027 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:48.895959 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12b61bf-96d6-45f6-b085-769a3bcf7c4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:48.896579 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12b61bf-96d6-45f6-b085-769a3bcf7c4a admin admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:48.896873 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12b61bf-96d6-45f6-b085-769a3bcf7c4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:48.897147 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12b61bf-96d6-45f6-b085-769a3bcf7c4a admin admin] PATH_INFO: `/v3/domains` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:48.897794 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12b61bf-96d6-45f6-b085-769a3bcf7c4a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:36:48.898096 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12b61bf-96d6-45f6-b085-769a3bcf7c4a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:48.899112 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: WARNING py.warnings [None req-d12b61bf-96d6-45f6-b085-769a3bcf7c4a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 24 17:36:48.899112 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: warnings.warn(msg) Aug 24 17:36:48.899112 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: Aug 24 17:36:48.900004 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12b61bf-96d6-45f6-b085-769a3bcf7c4a admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:48.911053 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 749/1496] 10.222.0.45 () {58 vars in 1153 bytes} [Wed Aug 24 17:36:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:36:48.919239 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22093bf-1495-4f9e-bb18-a50633408e37 None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:48.928866 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22093bf-1495-4f9e-bb18-a50633408e37 None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:48.988989 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:49.022664 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4811f05e-8123-4b60-af82-941904ef15c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:49.023264 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4811f05e-8123-4b60-af82-941904ef15c5 admin admin] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:49.023540 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4811f05e-8123-4b60-af82-941904ef15c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:49.023809 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4811f05e-8123-4b60-af82-941904ef15c5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:49.024433 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4811f05e-8123-4b60-af82-941904ef15c5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:49.026227 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: WARNING py.warnings [None req-4811f05e-8123-4b60-af82-941904ef15c5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 17:36:49.026227 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: warnings.warn(msg) Aug 24 17:36:49.026227 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: Aug 24 17:36:49.027423 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4811f05e-8123-4b60-af82-941904ef15c5 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:49.083405 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 748/1497] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 17:36:48 2022] POST /identity/v3/projects => generated 379 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:36:49.087641 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12b61bf-96d6-45f6-b085-769a3bcf7c4a admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:49.089150 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-d12b61bf-96d6-45f6-b085-769a3bcf7c4a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:36:49.122351 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12b61bf-96d6-45f6-b085-769a3bcf7c4a admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:49.213649 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5985941b-b08e-49d5-bd3e-a2e6313a0a81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:49.214251 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5985941b-b08e-49d5-bd3e-a2e6313a0a81 admin admin] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:49.214535 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5985941b-b08e-49d5-bd3e-a2e6313a0a81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:49.214805 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5985941b-b08e-49d5-bd3e-a2e6313a0a81 admin admin] PATH_INFO: `/v3/users` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:49.215435 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5985941b-b08e-49d5-bd3e-a2e6313a0a81 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:49.216445 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: WARNING py.warnings [None req-5985941b-b08e-49d5-bd3e-a2e6313a0a81 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 17:36:49.216445 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: warnings.warn(msg) Aug 24 17:36:49.216445 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: Aug 24 17:36:49.217687 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5985941b-b08e-49d5-bd3e-a2e6313a0a81 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:49.258947 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 750/1498] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 17:36:49 2022] POST /identity/v3/users => generated 290 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:36:49.262994 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4811f05e-8123-4b60-af82-941904ef15c5 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:49.264532 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-4811f05e-8123-4b60-af82-941904ef15c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:36:49.325126 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4811f05e-8123-4b60-af82-941904ef15c5 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:49.405496 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:49.417897 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a7e67d-b738-4c29-9c3e-b452fc567194 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:49.418507 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a7e67d-b738-4c29-9c3e-b452fc567194 admin admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:49.418786 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a7e67d-b738-4c29-9c3e-b452fc567194 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:49.419064 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a7e67d-b738-4c29-9c3e-b452fc567194 admin admin] PATH_INFO: `/v3/roles` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:49.419700 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a7e67d-b738-4c29-9c3e-b452fc567194 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:49.420739 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: WARNING py.warnings [None req-d5a7e67d-b738-4c29-9c3e-b452fc567194 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 24 17:36:49.420739 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: warnings.warn(msg) Aug 24 17:36:49.420739 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: Aug 24 17:36:49.421628 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a7e67d-b738-4c29-9c3e-b452fc567194 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:49.434377 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 749/1499] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:36:49 2022] GET /identity/v3/roles => generated 1420 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:36:49.438424 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5985941b-b08e-49d5-bd3e-a2e6313a0a81 admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:49.439942 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-5985941b-b08e-49d5-bd3e-a2e6313a0a81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:36:49.451281 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5985941b-b08e-49d5-bd3e-a2e6313a0a81 admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:49.476610 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:49.546472 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9645ffb-c7d5-4ef8-98b5-ec5aadcbde34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:49.547000 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9645ffb-c7d5-4ef8-98b5-ec5aadcbde34 admin admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:49.547274 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9645ffb-c7d5-4ef8-98b5-ec5aadcbde34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:49.547545 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9645ffb-c7d5-4ef8-98b5-ec5aadcbde34 admin admin] PATH_INFO: `/v3/roles` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:49.548184 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9645ffb-c7d5-4ef8-98b5-ec5aadcbde34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:49.549220 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: WARNING py.warnings [None req-a9645ffb-c7d5-4ef8-98b5-ec5aadcbde34 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 24 17:36:49.549220 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: warnings.warn(msg) Aug 24 17:36:49.549220 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: Aug 24 17:36:49.550105 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9645ffb-c7d5-4ef8-98b5-ec5aadcbde34 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:49.562281 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 751/1500] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:36:49 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:36:49.566581 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a7e67d-b738-4c29-9c3e-b452fc567194 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:49.568124 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-d5a7e67d-b738-4c29-9c3e-b452fc567194 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:36:49.602210 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a7e67d-b738-4c29-9c3e-b452fc567194 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:49.692879 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a7d049-6eb1-442f-b0f1-5550c0d7d3b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:49.693532 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a7d049-6eb1-442f-b0f1-5550c0d7d3b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:49.693809 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a7d049-6eb1-442f-b0f1-5550c0d7d3b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:49.694083 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a7d049-6eb1-442f-b0f1-5550c0d7d3b8 admin admin] PATH_INFO: `/v3/projects/e07c5e83d5cd481f8bf650505347e6ba/users/9a9f4fce904845a890ec69d6e44a5105/roles/02476a5b82eb4b188c18a5d706a3ef68` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:49.715490 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a7d049-6eb1-442f-b0f1-5550c0d7d3b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e07c5e83d5cd481f8bf650505347e6ba, user_id=9a9f4fce904845a890ec69d6e44a5105, role_id=02476a5b82eb4b188c18a5d706a3ef68)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:49.716517 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: WARNING py.warnings [None req-60a7d049-6eb1-442f-b0f1-5550c0d7d3b8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 17:36:49.716517 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: warnings.warn(msg) Aug 24 17:36:49.716517 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: Aug 24 17:36:49.718485 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a7d049-6eb1-442f-b0f1-5550c0d7d3b8 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:49.742295 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 750/1501] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 17:36:49 2022] PUT /identity/v3/projects/e07c5e83d5cd481f8bf650505347e6ba/users/9a9f4fce904845a890ec69d6e44a5105/roles/02476a5b82eb4b188c18a5d706a3ef68 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:36:49.754701 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360fa024-98b9-4701-b42f-674c67fa43d3 None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:49.754989 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360fa024-98b9-4701-b42f-674c67fa43d3 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:49.755267 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360fa024-98b9-4701-b42f-674c67fa43d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:49.811510 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-360fa024-98b9-4701-b42f-674c67fa43d3 None None] MFA Rules not processed for user `9a9f4fce904845a890ec69d6e44a5105`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:36:49.812752 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-360fa024-98b9-4701-b42f-674c67fa43d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:36:49.887998 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 752/1502] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:36:49 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:36:49.901702 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7460913b-c32e-4039-898d-864834648909 None None] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:49.901989 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7460913b-c32e-4039-898d-864834648909 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:49.902261 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7460913b-c32e-4039-898d-864834648909 None None] PATH_INFO: `/` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:49.903584 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 751/1503] 10.222.0.45 () {54 vars in 964 bytes} [Wed Aug 24 17:36:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:36:49.908196 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b42ca3-c412-4fbb-9670-6a76d0a576e7 None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:49.908534 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b42ca3-c412-4fbb-9670-6a76d0a576e7 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:49.908834 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b42ca3-c412-4fbb-9670-6a76d0a576e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:49.939777 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:49.984591 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-15b42ca3-c412-4fbb-9670-6a76d0a576e7 None None] MFA Rules not processed for user `59be342541264942ad87f60ff4de6c8e`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:36:49.985819 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-15b42ca3-c412-4fbb-9670-6a76d0a576e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:36:50.065986 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 753/1504] 10.222.0.45 () {58 vars in 1061 bytes} [Wed Aug 24 17:36:49 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:36:50.073866 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7460913b-c32e-4039-898d-864834648909 None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:50.075369 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-7460913b-c32e-4039-898d-864834648909 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:36:50.123730 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:50.178258 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7460913b-c32e-4039-898d-864834648909 None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:50.250078 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959b73f7-5ccc-4679-8529-2f6683a2a3dc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59be342541264942ad87f60ff4de6c8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:50.250680 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959b73f7-5ccc-4679-8529-2f6683a2a3dc service cinder] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:50.250954 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959b73f7-5ccc-4679-8529-2f6683a2a3dc service cinder] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:50.251341 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959b73f7-5ccc-4679-8529-2f6683a2a3dc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:50.253736 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-959b73f7-5ccc-4679-8529-2f6683a2a3dc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:36:50.350046 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959b73f7-5ccc-4679-8529-2f6683a2a3dc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:50.351418 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959b73f7-5ccc-4679-8529-2f6683a2a3dc service cinder] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:50.354871 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:50.369864 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 752/1505] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:36:50 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:36:50.414957 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4561a53-7a31-4092-8255-4b1281fb8e38 None None] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:50.415247 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4561a53-7a31-4092-8255-4b1281fb8e38 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:50.415524 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4561a53-7a31-4092-8255-4b1281fb8e38 None None] PATH_INFO: `/` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:50.416801 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 754/1506] 10.222.0.45 () {52 vars in 830 bytes} [Wed Aug 24 17:36:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:36:50.420430 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959b73f7-5ccc-4679-8529-2f6683a2a3dc service cinder] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:50.421701 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-959b73f7-5ccc-4679-8529-2f6683a2a3dc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:36:50.519200 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959b73f7-5ccc-4679-8529-2f6683a2a3dc service cinder] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:50.610921 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cb90c0-d21c-408a-915e-b9d3b0472eae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:50.612056 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cb90c0-d21c-408a-915e-b9d3b0472eae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:50.612358 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cb90c0-d21c-408a-915e-b9d3b0472eae admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:50.612657 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cb90c0-d21c-408a-915e-b9d3b0472eae admin admin] PATH_INFO: `/v3/users/9a9f4fce904845a890ec69d6e44a5105` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:50.624922 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cb90c0-d21c-408a-915e-b9d3b0472eae admin admin] RBAC: Authorizing `identity:delete_user(user_id=9a9f4fce904845a890ec69d6e44a5105)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:50.625941 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: WARNING py.warnings [None req-54cb90c0-d21c-408a-915e-b9d3b0472eae admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 17:36:50.625941 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: warnings.warn(msg) Aug 24 17:36:50.625941 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: Aug 24 17:36:50.627444 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cb90c0-d21c-408a-915e-b9d3b0472eae admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:50.636780 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:50.692099 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-54cb90c0-d21c-408a-915e-b9d3b0472eae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9a9f4fce904845a890ec69d6e44a5105'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:36:50.692570 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-54cb90c0-d21c-408a-915e-b9d3b0472eae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9a9f4fce904845a890ec69d6e44a5105'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:36:50.706860 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-54cb90c0-d21c-408a-915e-b9d3b0472eae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9a9f4fce904845a890ec69d6e44a5105'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:36:50.707422 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-54cb90c0-d21c-408a-915e-b9d3b0472eae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9a9f4fce904845a890ec69d6e44a5105'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:36:50.716109 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-54cb90c0-d21c-408a-915e-b9d3b0472eae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9a9f4fce904845a890ec69d6e44a5105'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:36:50.744545 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 753/1507] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 17:36:50 2022] DELETE /identity/v3/users/9a9f4fce904845a890ec69d6e44a5105 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:36:50.749014 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776825c5-9bb7-489a-8198-7b1cd59e3016 None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:50.749475 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776825c5-9bb7-489a-8198-7b1cd59e3016 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:50.749578 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776825c5-9bb7-489a-8198-7b1cd59e3016 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:50.805155 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-776825c5-9bb7-489a-8198-7b1cd59e3016 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:36:50.806349 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-776825c5-9bb7-489a-8198-7b1cd59e3016 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:36:50.881593 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:50.894556 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 755/1508] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:36:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:36:50.904165 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cb90c0-d21c-408a-915e-b9d3b0472eae admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:50.905458 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-54cb90c0-d21c-408a-915e-b9d3b0472eae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:36:51.004803 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cb90c0-d21c-408a-915e-b9d3b0472eae admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:51.057860 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:51.078469 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea1cf57-10af-4ed2-9b82-cbc0ea59a48a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43ad98a250a144099ccd18378aedd4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:51.079083 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea1cf57-10af-4ed2-9b82-cbc0ea59a48a service neutron] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:51.079362 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea1cf57-10af-4ed2-9b82-cbc0ea59a48a service neutron] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:51.079636 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea1cf57-10af-4ed2-9b82-cbc0ea59a48a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:51.081800 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-aea1cf57-10af-4ed2-9b82-cbc0ea59a48a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:36:51.189436 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea1cf57-10af-4ed2-9b82-cbc0ea59a48a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:51.190867 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea1cf57-10af-4ed2-9b82-cbc0ea59a48a service neutron] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:51.327225 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 754/1509] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:36:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:36:51.674872 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776825c5-9bb7-489a-8198-7b1cd59e3016 None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:51.676168 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-776825c5-9bb7-489a-8198-7b1cd59e3016 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:36:51.687287 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776825c5-9bb7-489a-8198-7b1cd59e3016 None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:51.787869 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0024ef37-1732-4077-b377-67c354a6371d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:51.788531 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0024ef37-1732-4077-b377-67c354a6371d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:51.788792 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0024ef37-1732-4077-b377-67c354a6371d admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:51.789063 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0024ef37-1732-4077-b377-67c354a6371d admin admin] PATH_INFO: `/v3/projects/e07c5e83d5cd481f8bf650505347e6ba` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:51.799457 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0024ef37-1732-4077-b377-67c354a6371d admin admin] RBAC: Authorizing `identity:delete_project(project_id=e07c5e83d5cd481f8bf650505347e6ba)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:51.800995 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: WARNING py.warnings [None req-0024ef37-1732-4077-b377-67c354a6371d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 17:36:51.800995 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: warnings.warn(msg) Aug 24 17:36:51.800995 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: Aug 24 17:36:51.801701 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0024ef37-1732-4077-b377-67c354a6371d admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:51.857413 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-0024ef37-1732-4077-b377-67c354a6371d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e07c5e83d5cd481f8bf650505347e6ba', 'request_id': 'req-0024ef37-1732-4077-b377-67c354a6371d'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:36:51.866394 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-0024ef37-1732-4077-b377-67c354a6371d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e07c5e83d5cd481f8bf650505347e6ba', 'request_id': 'req-0024ef37-1732-4077-b377-67c354a6371d'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:36:51.880633 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-0024ef37-1732-4077-b377-67c354a6371d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e07c5e83d5cd481f8bf650505347e6ba', 'request_id': 'req-0024ef37-1732-4077-b377-67c354a6371d'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:36:51.902428 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-0024ef37-1732-4077-b377-67c354a6371d admin admin] The token cache is being invalidate because project e07c5e83d5cd481f8bf650505347e6ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81758) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:36:51.902718 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-0024ef37-1732-4077-b377-67c354a6371d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:36:51.903232 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-0024ef37-1732-4077-b377-67c354a6371d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:36:51.904529 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 756/1510] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 17:36:51 2022] DELETE /identity/v3/projects/e07c5e83d5cd481f8bf650505347e6ba => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:37:01.324788 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=62 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:37:01.900464 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:38:18.150521 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144d34a8-53be-4cec-8f76-47d027392051 None None] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:38:18.151157 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144d34a8-53be-4cec-8f76-47d027392051 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:38:18.151157 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144d34a8-53be-4cec-8f76-47d027392051 None None] PATH_INFO: `/` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:38:18.152410 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 755/1511] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 17:38:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:38:18.157480 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb74e88-db97-4b75-b0db-33b4f8cad9f6 None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:38:18.157916 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb74e88-db97-4b75-b0db-33b4f8cad9f6 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:38:18.158014 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb74e88-db97-4b75-b0db-33b4f8cad9f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:38:18.244248 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-ddb74e88-db97-4b75-b0db-33b4f8cad9f6 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:38:18.245512 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-ddb74e88-db97-4b75-b0db-33b4f8cad9f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:38:18.350508 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 757/1512] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 17:38:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:38:18.502784 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144d34a8-53be-4cec-8f76-47d027392051 None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:38:18.505005 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-144d34a8-53be-4cec-8f76-47d027392051 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:38:18.622254 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144d34a8-53be-4cec-8f76-47d027392051 None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:38:18.627595 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e4e54f-2196-4cb4-be18-e00261f3258b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '053b7ce6d6f649ed91a26e434dab805e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:38:18.628193 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e4e54f-2196-4cb4-be18-e00261f3258b service nova] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:38:18.628490 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e4e54f-2196-4cb4-be18-e00261f3258b service nova] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:38:18.628792 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e4e54f-2196-4cb4-be18-e00261f3258b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:38:18.637540 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e4e54f-2196-4cb4-be18-e00261f3258b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:38:18.639181 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e4e54f-2196-4cb4-be18-e00261f3258b service nova] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:38:18.648680 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 756/1513] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:38:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:38:19.478397 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d140e5-12d8-4610-8fd7-c1b98215a469 None None] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:38:19.478956 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d140e5-12d8-4610-8fd7-c1b98215a469 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:38:19.478956 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d140e5-12d8-4610-8fd7-c1b98215a469 None None] PATH_INFO: `/` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:38:19.480310 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 758/1514] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 17:38:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:38:19.485087 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1a57d9-0ff1-4358-bf8e-0d455811bdbf None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:38:19.485523 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1a57d9-0ff1-4358-bf8e-0d455811bdbf None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:38:19.485634 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1a57d9-0ff1-4358-bf8e-0d455811bdbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:38:19.504203 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-ee1a57d9-0ff1-4358-bf8e-0d455811bdbf None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:38:19.505671 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-ee1a57d9-0ff1-4358-bf8e-0d455811bdbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:38:19.513297 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 757/1515] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 17:38:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:38:19.665078 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d140e5-12d8-4610-8fd7-c1b98215a469 None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:38:19.679670 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d140e5-12d8-4610-8fd7-c1b98215a469 None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:38:19.684960 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46676e25-eb28-4661-a768-7c9bf5115053 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '053b7ce6d6f649ed91a26e434dab805e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:38:19.685545 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46676e25-eb28-4661-a768-7c9bf5115053 service nova] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:38:19.685817 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46676e25-eb28-4661-a768-7c9bf5115053 service nova] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:38:19.686083 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46676e25-eb28-4661-a768-7c9bf5115053 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:38:19.693836 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46676e25-eb28-4661-a768-7c9bf5115053 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:38:19.695399 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46676e25-eb28-4661-a768-7c9bf5115053 service nova] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:38:19.704442 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 759/1516] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:38:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:38:20.758614 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725fb086-532b-4960-949d-ce5a9df33430 None None] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:38:20.759198 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725fb086-532b-4960-949d-ce5a9df33430 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:38:20.759198 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725fb086-532b-4960-949d-ce5a9df33430 None None] PATH_INFO: `/v3` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:38:20.760409 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 758/1517] 10.222.0.73 () {54 vars in 925 bytes} [Wed Aug 24 17:38:20 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:38:20.766410 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7156811a-d9b5-4bba-b7b9-4a4d6c7d31b5 None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:38:20.766836 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7156811a-d9b5-4bba-b7b9-4a4d6c7d31b5 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:38:20.766966 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7156811a-d9b5-4bba-b7b9-4a4d6c7d31b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:38:20.825912 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-7156811a-d9b5-4bba-b7b9-4a4d6c7d31b5 None None] MFA Rules not processed for user `37def4721b554b9ebb7704a19642d15b`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:38:20.827116 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-7156811a-d9b5-4bba-b7b9-4a4d6c7d31b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:38:20.843144 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:38:20.897134 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 760/1518] 10.222.0.73 () {58 vars in 1015 bytes} [Wed Aug 24 17:38:20 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:38:24.751321 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b805b02-e999-4fcd-b85b-d0b436b5364f None None] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:38:24.751876 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b805b02-e999-4fcd-b85b-d0b436b5364f None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:38:24.751876 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b805b02-e999-4fcd-b85b-d0b436b5364f None None] PATH_INFO: `/` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:38:24.753232 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 759/1519] 10.222.0.45 () {54 vars in 961 bytes} [Wed Aug 24 17:38:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:38:24.758334 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0125d2-f93f-47de-b441-383d0aa9c1be None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:38:24.758759 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0125d2-f93f-47de-b441-383d0aa9c1be None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:38:24.758872 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0125d2-f93f-47de-b441-383d0aa9c1be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:38:24.778597 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-dd0125d2-f93f-47de-b441-383d0aa9c1be None None] MFA Rules not processed for user `37def4721b554b9ebb7704a19642d15b`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:38:24.779777 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-dd0125d2-f93f-47de-b441-383d0aa9c1be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:38:24.788128 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 761/1520] 10.222.0.45 () {58 vars in 1058 bytes} [Wed Aug 24 17:38:24 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:38:24.796229 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b805b02-e999-4fcd-b85b-d0b436b5364f None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:38:24.809352 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b805b02-e999-4fcd-b85b-d0b436b5364f None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:38:24.814250 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff22e95-4f4e-4a62-8279-84348bbee56f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37def4721b554b9ebb7704a19642d15b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:38:24.814833 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff22e95-4f4e-4a62-8279-84348bbee56f service placement] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:38:24.815106 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff22e95-4f4e-4a62-8279-84348bbee56f service placement] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:38:24.815373 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff22e95-4f4e-4a62-8279-84348bbee56f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:38:24.823341 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff22e95-4f4e-4a62-8279-84348bbee56f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:38:24.824937 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff22e95-4f4e-4a62-8279-84348bbee56f service placement] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:38:24.834179 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 760/1521] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:38:24 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:38:30.924580 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6df6f2-7358-4c1c-923d-a2ec1d91c899 None None] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:38:30.925061 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6df6f2-7358-4c1c-923d-a2ec1d91c899 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:38:30.925172 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6df6f2-7358-4c1c-923d-a2ec1d91c899 None None] PATH_INFO: `/` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:38:30.926479 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 762/1522] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 17:38:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:38:30.931266 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29a7f4d-df0c-4dd3-99d4-df6eb910b7a1 None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:38:30.931650 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29a7f4d-df0c-4dd3-99d4-df6eb910b7a1 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:38:30.931810 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29a7f4d-df0c-4dd3-99d4-df6eb910b7a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:38:30.950407 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-e29a7f4d-df0c-4dd3-99d4-df6eb910b7a1 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:38:30.951582 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-e29a7f4d-df0c-4dd3-99d4-df6eb910b7a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:38:30.959237 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 761/1523] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 17:38:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:38:31.112417 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6df6f2-7358-4c1c-923d-a2ec1d91c899 None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:38:31.126901 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6df6f2-7358-4c1c-923d-a2ec1d91c899 None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:38:31.132649 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c4bd16-4bce-40c8-8a92-423b57edc681 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '053b7ce6d6f649ed91a26e434dab805e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:38:31.133248 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c4bd16-4bce-40c8-8a92-423b57edc681 service nova] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:38:31.133533 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c4bd16-4bce-40c8-8a92-423b57edc681 service nova] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:38:31.133809 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c4bd16-4bce-40c8-8a92-423b57edc681 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:38:31.142406 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c4bd16-4bce-40c8-8a92-423b57edc681 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:38:31.144003 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c4bd16-4bce-40c8-8a92-423b57edc681 service nova] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:38:31.154097 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 763/1524] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:38:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 17:38:31.967554 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638ca3a3-c278-4adf-a5c8-cb00c2b06522 None None] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:38:31.968125 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638ca3a3-c278-4adf-a5c8-cb00c2b06522 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:38:31.968125 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638ca3a3-c278-4adf-a5c8-cb00c2b06522 None None] PATH_INFO: `/` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:38:31.969483 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 762/1525] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 17:38:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:38:31.974235 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4cadd5-2edd-42f5-abad-cd71c6407169 None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:38:31.974631 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4cadd5-2edd-42f5-abad-cd71c6407169 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:38:31.974781 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4cadd5-2edd-42f5-abad-cd71c6407169 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:38:31.994870 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-8a4cadd5-2edd-42f5-abad-cd71c6407169 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:38:31.996069 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-8a4cadd5-2edd-42f5-abad-cd71c6407169 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:38:32.003901 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 764/1526] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 17:38:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:38:32.155398 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638ca3a3-c278-4adf-a5c8-cb00c2b06522 None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:38:32.170331 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638ca3a3-c278-4adf-a5c8-cb00c2b06522 None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:38:32.175734 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d039028-98c4-4419-9f4e-b6d93fafd076 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '053b7ce6d6f649ed91a26e434dab805e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:38:32.176321 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d039028-98c4-4419-9f4e-b6d93fafd076 service nova] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:38:32.176632 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d039028-98c4-4419-9f4e-b6d93fafd076 service nova] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:38:32.176908 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d039028-98c4-4419-9f4e-b6d93fafd076 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:38:32.184897 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d039028-98c4-4419-9f4e-b6d93fafd076 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:38:32.186445 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d039028-98c4-4419-9f4e-b6d93fafd076 service nova] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:38:32.195569 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 763/1527] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:38:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:38:35.944684 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad84cab-01fe-42d0-8a97-a3f1e6e3517e None None] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:38:35.945261 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad84cab-01fe-42d0-8a97-a3f1e6e3517e None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:38:35.945261 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad84cab-01fe-42d0-8a97-a3f1e6e3517e None None] PATH_INFO: `/` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:38:35.946586 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 765/1528] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 17:38:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:38:35.951705 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ea8d73-1c49-40ac-8b1a-34045d3db4e4 None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:38:35.952139 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ea8d73-1c49-40ac-8b1a-34045d3db4e4 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:38:35.952281 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ea8d73-1c49-40ac-8b1a-34045d3db4e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:38:35.971478 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-f6ea8d73-1c49-40ac-8b1a-34045d3db4e4 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:38:35.972699 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-f6ea8d73-1c49-40ac-8b1a-34045d3db4e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:38:35.980994 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 764/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 17:38:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:38:36.268804 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad84cab-01fe-42d0-8a97-a3f1e6e3517e None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:38:36.270588 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-dad84cab-01fe-42d0-8a97-a3f1e6e3517e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:38:36.347632 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad84cab-01fe-42d0-8a97-a3f1e6e3517e None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:38:36.351145 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d027cf2a-0403-41f1-a73a-50ab36b50d7a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43ad98a250a144099ccd18378aedd4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:38:36.351741 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d027cf2a-0403-41f1-a73a-50ab36b50d7a service neutron] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:38:36.352023 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d027cf2a-0403-41f1-a73a-50ab36b50d7a service neutron] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:38:36.352295 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d027cf2a-0403-41f1-a73a-50ab36b50d7a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:38:36.360960 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d027cf2a-0403-41f1-a73a-50ab36b50d7a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:38:36.362376 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d027cf2a-0403-41f1-a73a-50ab36b50d7a service neutron] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:38:36.371510 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 766/1530] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:38:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:38:38.280307 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36a312b-31b7-4c9c-925b-b966411b06a8 None None] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:38:38.280933 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36a312b-31b7-4c9c-925b-b966411b06a8 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:38:38.280933 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36a312b-31b7-4c9c-925b-b966411b06a8 None None] PATH_INFO: `/` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:38:38.282230 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 765/1531] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 17:38:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:38:38.287683 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ecd318-6515-4d76-b080-d2d5d84c265f None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:38:38.288134 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ecd318-6515-4d76-b080-d2d5d84c265f None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:38:38.288245 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ecd318-6515-4d76-b080-d2d5d84c265f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:38:38.307639 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-78ecd318-6515-4d76-b080-d2d5d84c265f None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:38:38.308837 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-78ecd318-6515-4d76-b080-d2d5d84c265f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:38:38.316957 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 767/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 17:38:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:38:38.606500 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36a312b-31b7-4c9c-925b-b966411b06a8 None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:38:38.608193 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-d36a312b-31b7-4c9c-925b-b966411b06a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:38:38.620886 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36a312b-31b7-4c9c-925b-b966411b06a8 None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:38:38.624259 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b39237-5190-4a64-8fe4-5e5d9db075db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43ad98a250a144099ccd18378aedd4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:38:38.624873 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b39237-5190-4a64-8fe4-5e5d9db075db service neutron] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:38:38.625148 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b39237-5190-4a64-8fe4-5e5d9db075db service neutron] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:38:38.625418 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b39237-5190-4a64-8fe4-5e5d9db075db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:38:38.634886 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b39237-5190-4a64-8fe4-5e5d9db075db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:38:38.636282 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b39237-5190-4a64-8fe4-5e5d9db075db service neutron] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:38:38.645460 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 766/1533] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:38:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:38:42.169303 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ea1561-7192-4808-afde-d232ef6c9d73 None None] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:38:42.169869 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ea1561-7192-4808-afde-d232ef6c9d73 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:38:42.169869 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ea1561-7192-4808-afde-d232ef6c9d73 None None] PATH_INFO: `/` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:38:42.171205 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 768/1534] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 17:38:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:38:42.176319 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f34357f-fb78-42f4-908b-34e9e1417974 None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:38:42.176764 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f34357f-fb78-42f4-908b-34e9e1417974 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:38:42.176873 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f34357f-fb78-42f4-908b-34e9e1417974 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:38:42.185059 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=81 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:38:42.198126 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-1f34357f-fb78-42f4-908b-34e9e1417974 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:38:42.199321 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-1f34357f-fb78-42f4-908b-34e9e1417974 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:38:42.207029 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 767/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 17:38:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:38:42.762513 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ea1561-7192-4808-afde-d232ef6c9d73 None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:38:42.764152 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-01ea1561-7192-4808-afde-d232ef6c9d73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:38:42.829115 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:38:42.842252 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ea1561-7192-4808-afde-d232ef6c9d73 None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:38:42.845734 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c040eb6-4079-4c12-9456-8a7dcf4b4f32 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59be342541264942ad87f60ff4de6c8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:38:42.846332 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c040eb6-4079-4c12-9456-8a7dcf4b4f32 service cinder] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:38:42.846606 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c040eb6-4079-4c12-9456-8a7dcf4b4f32 service cinder] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:38:42.846875 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c040eb6-4079-4c12-9456-8a7dcf4b4f32 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:38:42.855600 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c040eb6-4079-4c12-9456-8a7dcf4b4f32 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:38:42.857096 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c040eb6-4079-4c12-9456-8a7dcf4b4f32 service cinder] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:38:42.866225 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 769/1536] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:38:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:38:44.756601 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d294db5-9c3b-4664-9214-9f55d8929de5 None None] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:38:44.757177 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d294db5-9c3b-4664-9214-9f55d8929de5 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:38:44.757319 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d294db5-9c3b-4664-9214-9f55d8929de5 None None] PATH_INFO: `/` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:38:44.758704 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 768/1537] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 17:38:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:38:44.764012 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be83df21-6144-48c8-84d0-164e228b372c None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:38:44.764419 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be83df21-6144-48c8-84d0-164e228b372c None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:38:44.764588 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be83df21-6144-48c8-84d0-164e228b372c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:38:44.784653 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-be83df21-6144-48c8-84d0-164e228b372c None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:38:44.785840 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-be83df21-6144-48c8-84d0-164e228b372c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:38:44.794091 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 770/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 17:38:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:38:45.362784 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d294db5-9c3b-4664-9214-9f55d8929de5 None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:38:45.364439 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-3d294db5-9c3b-4664-9214-9f55d8929de5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:38:45.377389 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d294db5-9c3b-4664-9214-9f55d8929de5 None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:38:45.380843 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3a9452-e220-4931-b3d5-b34b818e7a73 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59be342541264942ad87f60ff4de6c8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:38:45.381790 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3a9452-e220-4931-b3d5-b34b818e7a73 service cinder] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:38:45.382067 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3a9452-e220-4931-b3d5-b34b818e7a73 service cinder] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:38:45.382338 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3a9452-e220-4931-b3d5-b34b818e7a73 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:38:45.391658 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3a9452-e220-4931-b3d5-b34b818e7a73 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:38:45.393133 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3a9452-e220-4931-b3d5-b34b818e7a73 service cinder] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:38:45.403262 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 769/1539] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:38:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:38:54.778544 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=11 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:38:55.398411 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=8 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:40:07.306047 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d805afa2-95f0-4f78-8803-53b1ab12ab2c None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:07.306696 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d805afa2-95f0-4f78-8803-53b1ab12ab2c None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:07.306696 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d805afa2-95f0-4f78-8803-53b1ab12ab2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:07.321904 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a860ce1-f4d0-4c61-a366-34c7670b32e9 None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:07.322479 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a860ce1-f4d0-4c61-a366-34c7670b32e9 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:07.322479 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a860ce1-f4d0-4c61-a366-34c7670b32e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:07.337576 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-d805afa2-95f0-4f78-8803-53b1ab12ab2c None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:40:07.338790 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-d805afa2-95f0-4f78-8803-53b1ab12ab2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:07.351622 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-0a860ce1-f4d0-4c61-a366-34c7670b32e9 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:40:07.353103 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-0a860ce1-f4d0-4c61-a366-34c7670b32e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:07.354588 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 771/1540] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:40:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:40:07.367753 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 770/1541] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:40:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:40:07.372653 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1c0d12-a9fd-49d6-85cf-0748bf234418 None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:07.372942 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1c0d12-a9fd-49d6-85cf-0748bf234418 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:07.373219 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1c0d12-a9fd-49d6-85cf-0748bf234418 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:07.378273 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a493cd5b-4544-4598-a50a-47a2db35264d None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:07.378570 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a493cd5b-4544-4598-a50a-47a2db35264d None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:07.378845 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a493cd5b-4544-4598-a50a-47a2db35264d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:07.409766 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-4b1c0d12-a9fd-49d6-85cf-0748bf234418 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:40:07.411138 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-4b1c0d12-a9fd-49d6-85cf-0748bf234418 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:07.412411 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-a493cd5b-4544-4598-a50a-47a2db35264d None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:40:07.413611 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-a493cd5b-4544-4598-a50a-47a2db35264d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:07.430410 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 772/1542] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:40:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:40:07.430614 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 771/1543] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:40:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:40:07.435195 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1c0d12-a9fd-49d6-85cf-0748bf234418 None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:07.435827 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a493cd5b-4544-4598-a50a-47a2db35264d None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:07.449882 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a493cd5b-4544-4598-a50a-47a2db35264d None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:07.450952 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1c0d12-a9fd-49d6-85cf-0748bf234418 None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:07.462276 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cce2f9c-62c7-424c-a828-3da60532a7f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:07.463113 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cce2f9c-62c7-424c-a828-3da60532a7f5 admin admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:07.463386 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cce2f9c-62c7-424c-a828-3da60532a7f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:07.464079 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6bf9c4-154d-41eb-b18b-838da39db456 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:07.464323 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cce2f9c-62c7-424c-a828-3da60532a7f5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:07.464713 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6bf9c4-154d-41eb-b18b-838da39db456 admin admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:07.464991 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6bf9c4-154d-41eb-b18b-838da39db456 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:07.465112 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cce2f9c-62c7-424c-a828-3da60532a7f5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:40:07.465313 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6bf9c4-154d-41eb-b18b-838da39db456 admin admin] PATH_INFO: `/v3/domains` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:07.465439 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cce2f9c-62c7-424c-a828-3da60532a7f5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:07.465903 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6bf9c4-154d-41eb-b18b-838da39db456 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:40:07.466204 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6bf9c4-154d-41eb-b18b-838da39db456 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:07.466319 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: WARNING py.warnings [None req-3cce2f9c-62c7-424c-a828-3da60532a7f5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 24 17:40:07.466319 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: warnings.warn(msg) Aug 24 17:40:07.466319 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: Aug 24 17:40:07.467193 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cce2f9c-62c7-424c-a828-3da60532a7f5 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:07.467781 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6bf9c4-154d-41eb-b18b-838da39db456 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:07.486947 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 772/1544] 10.222.0.45 () {58 vars in 1153 bytes} [Wed Aug 24 17:40:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:40:07.489559 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 773/1545] 10.222.0.45 () {58 vars in 1153 bytes} [Wed Aug 24 17:40:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:40:07.495458 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cce2f9c-62c7-424c-a828-3da60532a7f5 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:07.502077 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6bf9c4-154d-41eb-b18b-838da39db456 admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:07.508030 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cce2f9c-62c7-424c-a828-3da60532a7f5 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:07.517943 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06750838-b967-42aa-8eed-b900d3a3f417 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:07.518895 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06750838-b967-42aa-8eed-b900d3a3f417 admin admin] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:07.519492 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06750838-b967-42aa-8eed-b900d3a3f417 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:07.520126 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06750838-b967-42aa-8eed-b900d3a3f417 admin admin] PATH_INFO: `/v3/projects` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:07.521064 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06750838-b967-42aa-8eed-b900d3a3f417 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:07.523081 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6bf9c4-154d-41eb-b18b-838da39db456 admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:07.523457 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06750838-b967-42aa-8eed-b900d3a3f417 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:07.549303 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b69ce7-cb68-4107-90f5-a9c0202279ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:07.550020 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b69ce7-cb68-4107-90f5-a9c0202279ca admin admin] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:07.550255 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b69ce7-cb68-4107-90f5-a9c0202279ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:07.550470 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b69ce7-cb68-4107-90f5-a9c0202279ca admin admin] PATH_INFO: `/v3/projects` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:07.551113 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b69ce7-cb68-4107-90f5-a9c0202279ca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:07.552210 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: WARNING py.warnings [None req-05b69ce7-cb68-4107-90f5-a9c0202279ca admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 17:40:07.552210 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: warnings.warn(msg) Aug 24 17:40:07.552210 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: Aug 24 17:40:07.553502 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b69ce7-cb68-4107-90f5-a9c0202279ca admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:07.599438 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 773/1546] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 17:40:07 2022] POST /identity/v3/projects => generated 403 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:40:07.605893 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06750838-b967-42aa-8eed-b900d3a3f417 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:07.623369 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 774/1547] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 17:40:07 2022] POST /identity/v3/projects => generated 365 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:40:07.623762 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b69ce7-cb68-4107-90f5-a9c0202279ca admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:07.753548 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b69ce7-cb68-4107-90f5-a9c0202279ca admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:07.757130 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06750838-b967-42aa-8eed-b900d3a3f417 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:07.769780 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57b816e-bc92-4401-9091-2c242a23c385 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:07.770687 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57b816e-bc92-4401-9091-2c242a23c385 admin admin] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:07.771085 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57b816e-bc92-4401-9091-2c242a23c385 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:07.771467 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57b816e-bc92-4401-9091-2c242a23c385 admin admin] PATH_INFO: `/v3/users` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:07.771851 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a693ef6-41ee-4857-8903-0ef32e2c69f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:07.772338 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57b816e-bc92-4401-9091-2c242a23c385 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:07.772866 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a693ef6-41ee-4857-8903-0ef32e2c69f1 admin admin] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:07.773321 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a693ef6-41ee-4857-8903-0ef32e2c69f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:07.773747 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a693ef6-41ee-4857-8903-0ef32e2c69f1 admin admin] PATH_INFO: `/v3/users` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:07.774526 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a693ef6-41ee-4857-8903-0ef32e2c69f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:07.774840 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57b816e-bc92-4401-9091-2c242a23c385 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:07.776125 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: WARNING py.warnings [None req-3a693ef6-41ee-4857-8903-0ef32e2c69f1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 17:40:07.776125 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: warnings.warn(msg) Aug 24 17:40:07.776125 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: Aug 24 17:40:07.783503 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a693ef6-41ee-4857-8903-0ef32e2c69f1 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:07.830196 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 774/1548] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 17:40:07 2022] POST /identity/v3/users => generated 302 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:40:07.831551 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1b1417-0cd5-47a8-9718-deff81309165 None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:07.831843 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1b1417-0cd5-47a8-9718-deff81309165 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:07.832312 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1b1417-0cd5-47a8-9718-deff81309165 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:07.839203 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 775/1549] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 17:40:07 2022] POST /identity/v3/users => generated 283 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:40:07.840570 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a79de9-e03f-48bb-899e-74dcd0c8ae33 None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:07.840934 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a79de9-e03f-48bb-899e-74dcd0c8ae33 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:07.841281 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a79de9-e03f-48bb-899e-74dcd0c8ae33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:07.862892 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-7f1b1417-0cd5-47a8-9718-deff81309165 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:40:07.864457 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-7f1b1417-0cd5-47a8-9718-deff81309165 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:07.873866 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-e5a79de9-e03f-48bb-899e-74dcd0c8ae33 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:40:07.874196 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 775/1550] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:40:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:40:07.875159 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-e5a79de9-e03f-48bb-899e-74dcd0c8ae33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:07.876101 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc0d9f9-94b5-488f-aebd-243f64a4697c None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:07.876702 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc0d9f9-94b5-488f-aebd-243f64a4697c None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:07.877285 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc0d9f9-94b5-488f-aebd-243f64a4697c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:07.894352 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 776/1551] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:40:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:40:07.895241 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a79de9-e03f-48bb-899e-74dcd0c8ae33 None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:07.900176 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-dcc0d9f9-94b5-488f-aebd-243f64a4697c None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:40:07.901670 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-dcc0d9f9-94b5-488f-aebd-243f64a4697c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:07.913117 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a79de9-e03f-48bb-899e-74dcd0c8ae33 None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:07.914549 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 776/1552] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:40:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:40:07.916042 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc0d9f9-94b5-488f-aebd-243f64a4697c None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:07.918451 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957033d1-5864-43e7-ac0e-0367d975dfd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:07.919135 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957033d1-5864-43e7-ac0e-0367d975dfd3 admin admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:07.919477 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957033d1-5864-43e7-ac0e-0367d975dfd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:07.919814 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957033d1-5864-43e7-ac0e-0367d975dfd3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:07.920532 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957033d1-5864-43e7-ac0e-0367d975dfd3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:07.922391 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957033d1-5864-43e7-ac0e-0367d975dfd3 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:07.929389 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc0d9f9-94b5-488f-aebd-243f64a4697c None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:07.937698 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c25b534-9751-412f-8caa-f1601a47c923 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:07.938506 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c25b534-9751-412f-8caa-f1601a47c923 admin admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:07.939028 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c25b534-9751-412f-8caa-f1601a47c923 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:07.939579 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c25b534-9751-412f-8caa-f1601a47c923 admin admin] PATH_INFO: `/v3/roles` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:07.939966 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 777/1553] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:40:07 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:40:07.940603 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c25b534-9751-412f-8caa-f1601a47c923 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:07.941523 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbb7bb8-c6d1-4c03-9dde-4e0fb3eea6ab None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:07.941903 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbb7bb8-c6d1-4c03-9dde-4e0fb3eea6ab None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:07.942274 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbb7bb8-c6d1-4c03-9dde-4e0fb3eea6ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:07.943472 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c25b534-9751-412f-8caa-f1601a47c923 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:07.958554 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 777/1554] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:40:07 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:40:07.960181 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fef8d7-5d49-47f6-8421-be96b21281a1 None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:07.960668 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fef8d7-5d49-47f6-8421-be96b21281a1 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:07.961241 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fef8d7-5d49-47f6-8421-be96b21281a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:07.961967 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-9fbb7bb8-c6d1-4c03-9dde-4e0fb3eea6ab None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:40:07.963166 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-9fbb7bb8-c6d1-4c03-9dde-4e0fb3eea6ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:07.976264 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 778/1555] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:40:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:40:07.976264 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75612ea2-8226-4b5f-9ac4-395c34f1ff9b None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:07.976264 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75612ea2-8226-4b5f-9ac4-395c34f1ff9b None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:07.976264 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75612ea2-8226-4b5f-9ac4-395c34f1ff9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:07.990034 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-49fef8d7-5d49-47f6-8421-be96b21281a1 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:40:07.991579 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-49fef8d7-5d49-47f6-8421-be96b21281a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:07.999485 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 778/1556] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:40:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:40:08.000825 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d6645d-c28c-4ee3-86ea-74ec1e6e383c None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:08.001107 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d6645d-c28c-4ee3-86ea-74ec1e6e383c None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:08.001424 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d6645d-c28c-4ee3-86ea-74ec1e6e383c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:08.002699 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-75612ea2-8226-4b5f-9ac4-395c34f1ff9b None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:40:08.003945 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-75612ea2-8226-4b5f-9ac4-395c34f1ff9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:08.012236 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 779/1557] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:40:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:40:08.013137 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75612ea2-8226-4b5f-9ac4-395c34f1ff9b None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:08.021574 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-64d6645d-c28c-4ee3-86ea-74ec1e6e383c None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:40:08.022757 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-64d6645d-c28c-4ee3-86ea-74ec1e6e383c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:08.022881 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75612ea2-8226-4b5f-9ac4-395c34f1ff9b None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:08.027267 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360c654e-add4-4625-8025-568a5201d95e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:08.027845 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360c654e-add4-4625-8025-568a5201d95e admin admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:08.028119 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360c654e-add4-4625-8025-568a5201d95e admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:08.028390 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360c654e-add4-4625-8025-568a5201d95e admin admin] PATH_INFO: `/v3/roles` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:08.029060 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360c654e-add4-4625-8025-568a5201d95e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:08.030530 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 779/1558] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:40:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:40:08.030646 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360c654e-add4-4625-8025-568a5201d95e admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:08.031693 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d737cb9-ec26-45e8-903c-662b68440cef None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:08.031979 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d737cb9-ec26-45e8-903c-662b68440cef None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:08.032249 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d737cb9-ec26-45e8-903c-662b68440cef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:08.046599 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 780/1559] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:40:08 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:40:08.046599 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360c654e-add4-4625-8025-568a5201d95e admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:08.052065 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-6d737cb9-ec26-45e8-903c-662b68440cef None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:40:08.053275 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-6d737cb9-ec26-45e8-903c-662b68440cef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:08.053728 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360c654e-add4-4625-8025-568a5201d95e admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:08.058334 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae864d08-5e41-4be2-b752-2d24452681f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:08.058945 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae864d08-5e41-4be2-b752-2d24452681f0 admin admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:08.059357 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae864d08-5e41-4be2-b752-2d24452681f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:08.059640 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae864d08-5e41-4be2-b752-2d24452681f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:08.060274 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae864d08-5e41-4be2-b752-2d24452681f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:08.060940 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 780/1560] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:40:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:40:08.061964 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae864d08-5e41-4be2-b752-2d24452681f0 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:08.062080 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83c0b34-b037-40c5-959d-71c3454e9407 None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:08.062361 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83c0b34-b037-40c5-959d-71c3454e9407 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:08.062633 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83c0b34-b037-40c5-959d-71c3454e9407 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:08.073515 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 781/1561] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:40:08 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:40:08.075110 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4962ff15-9cd4-4e53-8496-b7a2f4b9b0f2 None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:08.075400 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4962ff15-9cd4-4e53-8496-b7a2f4b9b0f2 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:08.075672 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4962ff15-9cd4-4e53-8496-b7a2f4b9b0f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:08.087053 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-b83c0b34-b037-40c5-959d-71c3454e9407 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:40:08.088254 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-b83c0b34-b037-40c5-959d-71c3454e9407 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:08.094944 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-4962ff15-9cd4-4e53-8496-b7a2f4b9b0f2 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:40:08.096301 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-4962ff15-9cd4-4e53-8496-b7a2f4b9b0f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:08.096416 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 781/1562] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:40:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:40:08.097005 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83c0b34-b037-40c5-959d-71c3454e9407 None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:08.105984 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83c0b34-b037-40c5-959d-71c3454e9407 None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:08.107127 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 782/1563] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:40:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:40:08.107924 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4962ff15-9cd4-4e53-8496-b7a2f4b9b0f2 None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:08.110885 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978363ce-e811-4d55-9f9c-933badae8843 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:08.111476 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978363ce-e811-4d55-9f9c-933badae8843 admin admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:08.111752 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978363ce-e811-4d55-9f9c-933badae8843 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:08.112021 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978363ce-e811-4d55-9f9c-933badae8843 admin admin] PATH_INFO: `/v3/domains` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:08.112689 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978363ce-e811-4d55-9f9c-933badae8843 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:40:08.113043 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978363ce-e811-4d55-9f9c-933badae8843 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:08.114726 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978363ce-e811-4d55-9f9c-933badae8843 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:08.118122 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4962ff15-9cd4-4e53-8496-b7a2f4b9b0f2 None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:08.122899 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac848df-791e-4746-8188-cd5eb6544d81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:08.123494 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac848df-791e-4746-8188-cd5eb6544d81 admin admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:08.123773 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac848df-791e-4746-8188-cd5eb6544d81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:08.124071 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac848df-791e-4746-8188-cd5eb6544d81 admin admin] PATH_INFO: `/v3/domains` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:08.124743 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac848df-791e-4746-8188-cd5eb6544d81 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:40:08.125048 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac848df-791e-4746-8188-cd5eb6544d81 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:08.126193 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 782/1564] 10.222.0.45 () {58 vars in 1153 bytes} [Wed Aug 24 17:40:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:40:08.127327 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac848df-791e-4746-8188-cd5eb6544d81 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:08.127722 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978363ce-e811-4d55-9f9c-933badae8843 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:08.137915 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978363ce-e811-4d55-9f9c-933badae8843 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:08.139675 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 783/1565] 10.222.0.45 () {58 vars in 1153 bytes} [Wed Aug 24 17:40:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:40:08.140666 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac848df-791e-4746-8188-cd5eb6544d81 admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:08.143093 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0e9b18-b14a-4883-9918-9992a6c99666 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:08.143685 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0e9b18-b14a-4883-9918-9992a6c99666 admin admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:08.143966 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0e9b18-b14a-4883-9918-9992a6c99666 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:08.144239 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0e9b18-b14a-4883-9918-9992a6c99666 admin admin] PATH_INFO: `/v3/domains` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:08.144916 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0e9b18-b14a-4883-9918-9992a6c99666 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:40:08.145219 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0e9b18-b14a-4883-9918-9992a6c99666 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:08.146808 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0e9b18-b14a-4883-9918-9992a6c99666 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:08.151133 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac848df-791e-4746-8188-cd5eb6544d81 admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:08.156079 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9e2695-ec52-47a1-ac65-677ee1d5f3cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:08.156740 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9e2695-ec52-47a1-ac65-677ee1d5f3cd admin admin] REQUEST_METHOD: `PUT` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:08.157018 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9e2695-ec52-47a1-ac65-677ee1d5f3cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:08.157294 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9e2695-ec52-47a1-ac65-677ee1d5f3cd admin admin] PATH_INFO: `/v3/projects/acc9a400b1364f3eb1187796992fa846/users/f10fd8e06543489eb7bd0858d8cb9f34/roles/02476a5b82eb4b188c18a5d706a3ef68` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:08.165559 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 783/1566] 10.222.0.45 () {58 vars in 1153 bytes} [Wed Aug 24 17:40:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:40:08.165559 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95502128-6344-4611-8c93-1f4910c0d3f8 None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:08.165559 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95502128-6344-4611-8c93-1f4910c0d3f8 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:08.165559 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95502128-6344-4611-8c93-1f4910c0d3f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:08.178308 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-95502128-6344-4611-8c93-1f4910c0d3f8 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:40:08.178609 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9e2695-ec52-47a1-ac65-677ee1d5f3cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=acc9a400b1364f3eb1187796992fa846, user_id=f10fd8e06543489eb7bd0858d8cb9f34, role_id=02476a5b82eb4b188c18a5d706a3ef68)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:08.179492 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-95502128-6344-4611-8c93-1f4910c0d3f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:08.179640 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: WARNING py.warnings [None req-0b9e2695-ec52-47a1-ac65-677ee1d5f3cd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 17:40:08.179640 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: warnings.warn(msg) Aug 24 17:40:08.179640 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: Aug 24 17:40:08.181604 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9e2695-ec52-47a1-ac65-677ee1d5f3cd admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:08.187879 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 784/1567] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:40:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:40:08.188452 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95502128-6344-4611-8c93-1f4910c0d3f8 None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:08.197812 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 784/1568] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 17:40:08 2022] PUT /identity/v3/projects/acc9a400b1364f3eb1187796992fa846/users/f10fd8e06543489eb7bd0858d8cb9f34/roles/02476a5b82eb4b188c18a5d706a3ef68 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:40:08.198302 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95502128-6344-4611-8c93-1f4910c0d3f8 None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:08.198986 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9e2695-ec52-47a1-ac65-677ee1d5f3cd admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:08.203844 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:40:08.211227 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:40:08.263177 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c352c5-9e8c-4149-b88f-5abf60536143 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:08.263845 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c352c5-9e8c-4149-b88f-5abf60536143 admin admin] REQUEST_METHOD: `PUT` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:08.264124 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c352c5-9e8c-4149-b88f-5abf60536143 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:08.264399 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c352c5-9e8c-4149-b88f-5abf60536143 admin admin] PATH_INFO: `/v3/projects/a6550d4a026d481a9a81820075c5c4b5/users/d2a240ceae7d424f8c2bda4b534eeb98/roles/02476a5b82eb4b188c18a5d706a3ef68` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:08.274617 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9e2695-ec52-47a1-ac65-677ee1d5f3cd admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:08.279427 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49feecc-8155-4758-aa7f-dc144eb2fca1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:08.279594 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c352c5-9e8c-4149-b88f-5abf60536143 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6550d4a026d481a9a81820075c5c4b5, user_id=d2a240ceae7d424f8c2bda4b534eeb98, role_id=02476a5b82eb4b188c18a5d706a3ef68)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:08.280032 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49feecc-8155-4758-aa7f-dc144eb2fca1 admin admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:08.280308 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49feecc-8155-4758-aa7f-dc144eb2fca1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:08.280631 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49feecc-8155-4758-aa7f-dc144eb2fca1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:08.281255 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49feecc-8155-4758-aa7f-dc144eb2fca1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:40:08.281559 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49feecc-8155-4758-aa7f-dc144eb2fca1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:08.282633 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c352c5-9e8c-4149-b88f-5abf60536143 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:08.283256 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49feecc-8155-4758-aa7f-dc144eb2fca1 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:08.295022 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 785/1569] 10.222.0.45 () {58 vars in 1153 bytes} [Wed Aug 24 17:40:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:40:08.301315 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 785/1570] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 17:40:08 2022] PUT /identity/v3/projects/a6550d4a026d481a9a81820075c5c4b5/users/d2a240ceae7d424f8c2bda4b534eeb98/roles/02476a5b82eb4b188c18a5d706a3ef68 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:40:08.301315 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c352c5-9e8c-4149-b88f-5abf60536143 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:08.301578 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cb7d4d-2d2b-4f24-9c82-056a73b36950 None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:08.301578 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cb7d4d-2d2b-4f24-9c82-056a73b36950 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:08.301578 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cb7d4d-2d2b-4f24-9c82-056a73b36950 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:08.318888 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-85cb7d4d-2d2b-4f24-9c82-056a73b36950 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:40:08.320086 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-85cb7d4d-2d2b-4f24-9c82-056a73b36950 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:08.363747 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c352c5-9e8c-4149-b88f-5abf60536143 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:08.368403 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b29edb-d62c-4a7f-ab87-0522e8fbe5be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:08.369130 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b29edb-d62c-4a7f-ab87-0522e8fbe5be admin admin] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:08.369474 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b29edb-d62c-4a7f-ab87-0522e8fbe5be admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:08.369752 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b29edb-d62c-4a7f-ab87-0522e8fbe5be admin admin] PATH_INFO: `/v3/projects` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:08.370401 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b29edb-d62c-4a7f-ab87-0522e8fbe5be admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:08.372411 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b29edb-d62c-4a7f-ab87-0522e8fbe5be admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:08.378731 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 786/1571] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:40:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:40:08.379604 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cb7d4d-2d2b-4f24-9c82-056a73b36950 None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:08.388774 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cb7d4d-2d2b-4f24-9c82-056a73b36950 None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:08.394816 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf958223-e0c4-4be7-be73-45bcb5976244 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:08.395465 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf958223-e0c4-4be7-be73-45bcb5976244 admin admin] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:08.395740 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf958223-e0c4-4be7-be73-45bcb5976244 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:08.396011 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf958223-e0c4-4be7-be73-45bcb5976244 admin admin] PATH_INFO: `/v3/projects` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:08.396653 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf958223-e0c4-4be7-be73-45bcb5976244 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:08.398537 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf958223-e0c4-4be7-be73-45bcb5976244 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:08.405441 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 786/1572] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 17:40:08 2022] POST /identity/v3/projects => generated 383 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:40:08.405441 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b29edb-d62c-4a7f-ab87-0522e8fbe5be admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:08.427133 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 787/1573] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 17:40:08 2022] POST /identity/v3/projects => generated 387 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:40:08.428255 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf958223-e0c4-4be7-be73-45bcb5976244 admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:08.469552 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b29edb-d62c-4a7f-ab87-0522e8fbe5be admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:08.489396 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:40:08.490500 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf958223-e0c4-4be7-be73-45bcb5976244 admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:08.495857 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc80b886-5017-4da1-9584-ad0a42902e4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:08.496638 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc80b886-5017-4da1-9584-ad0a42902e4b admin admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:08.496937 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc80b886-5017-4da1-9584-ad0a42902e4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:08.497218 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc80b886-5017-4da1-9584-ad0a42902e4b admin admin] PATH_INFO: `/v3/domains` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:08.497865 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc80b886-5017-4da1-9584-ad0a42902e4b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:40:08.498173 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc80b886-5017-4da1-9584-ad0a42902e4b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:08.499780 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc80b886-5017-4da1-9584-ad0a42902e4b admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:08.512199 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 788/1574] 10.222.0.45 () {58 vars in 1153 bytes} [Wed Aug 24 17:40:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:40:08.513431 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd081f7-75f1-44ba-a464-dfb21265c57e None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:08.513723 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd081f7-75f1-44ba-a464-dfb21265c57e None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:08.514000 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd081f7-75f1-44ba-a464-dfb21265c57e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:08.527544 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8224e8-0088-4149-994c-4328772c5347 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:08.528159 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8224e8-0088-4149-994c-4328772c5347 admin admin] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:08.528438 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8224e8-0088-4149-994c-4328772c5347 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:08.528757 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8224e8-0088-4149-994c-4328772c5347 admin admin] PATH_INFO: `/v3/projects` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:08.529385 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8224e8-0088-4149-994c-4328772c5347 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:08.531286 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8224e8-0088-4149-994c-4328772c5347 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:08.535867 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:40:08.543961 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-6fd081f7-75f1-44ba-a464-dfb21265c57e None None] MFA Rules not processed for user `f10fd8e06543489eb7bd0858d8cb9f34`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:40:08.545754 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-6fd081f7-75f1-44ba-a464-dfb21265c57e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:08.558554 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 787/1575] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 17:40:08 2022] POST /identity/v3/projects => generated 393 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:40:08.559370 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8224e8-0088-4149-994c-4328772c5347 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:08.614464 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 789/1576] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:40:08 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:40:08.615637 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f94c84d-91a7-46d2-818c-2fcb4b05d81c None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:08.615935 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f94c84d-91a7-46d2-818c-2fcb4b05d81c None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:08.616243 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f94c84d-91a7-46d2-818c-2fcb4b05d81c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:08.624471 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8224e8-0088-4149-994c-4328772c5347 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:08.629682 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3076e9f5-3627-45f2-831f-0f84a1511941 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:08.630279 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3076e9f5-3627-45f2-831f-0f84a1511941 admin admin] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:08.630555 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3076e9f5-3627-45f2-831f-0f84a1511941 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:08.630825 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3076e9f5-3627-45f2-831f-0f84a1511941 admin admin] PATH_INFO: `/v3/projects` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:08.631439 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3076e9f5-3627-45f2-831f-0f84a1511941 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:08.633354 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3076e9f5-3627-45f2-831f-0f84a1511941 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:08.643159 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-4f94c84d-91a7-46d2-818c-2fcb4b05d81c None None] MFA Rules not processed for user `d2a240ceae7d424f8c2bda4b534eeb98`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:40:08.644366 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-4f94c84d-91a7-46d2-818c-2fcb4b05d81c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:08.660095 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 788/1577] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 17:40:08 2022] POST /identity/v3/projects => generated 379 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:40:08.661318 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3076e9f5-3627-45f2-831f-0f84a1511941 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:08.712486 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 790/1578] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:40:08 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:40:08.713348 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f94c84d-91a7-46d2-818c-2fcb4b05d81c None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:08.723492 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f94c84d-91a7-46d2-818c-2fcb4b05d81c None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:08.725272 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3076e9f5-3627-45f2-831f-0f84a1511941 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:08.728041 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c24598-036b-4223-8bfa-8e9fd8a916be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:08.728671 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c24598-036b-4223-8bfa-8e9fd8a916be admin admin] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:08.728955 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c24598-036b-4223-8bfa-8e9fd8a916be admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:08.729233 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c24598-036b-4223-8bfa-8e9fd8a916be admin admin] PATH_INFO: `/v3/users` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:08.729876 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a96369a-ffac-4c70-bbd1-ec31183909c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:08.730004 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c24598-036b-4223-8bfa-8e9fd8a916be admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:08.730461 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a96369a-ffac-4c70-bbd1-ec31183909c8 admin admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:08.730736 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a96369a-ffac-4c70-bbd1-ec31183909c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:08.731007 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a96369a-ffac-4c70-bbd1-ec31183909c8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:08.731645 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a96369a-ffac-4c70-bbd1-ec31183909c8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:40:08.731886 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c24598-036b-4223-8bfa-8e9fd8a916be admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:08.731995 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a96369a-ffac-4c70-bbd1-ec31183909c8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:08.733573 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a96369a-ffac-4c70-bbd1-ec31183909c8 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:08.746429 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 789/1579] 10.222.0.45 () {58 vars in 1153 bytes} [Wed Aug 24 17:40:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:40:08.747354 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a96369a-ffac-4c70-bbd1-ec31183909c8 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:08.757390 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a96369a-ffac-4c70-bbd1-ec31183909c8 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:08.760599 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 791/1580] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 17:40:08 2022] POST /identity/v3/users => generated 292 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:40:08.761406 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c24598-036b-4223-8bfa-8e9fd8a916be admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:08.762447 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acb0e51-998b-4462-ba52-a4830e424895 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:08.763037 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acb0e51-998b-4462-ba52-a4830e424895 admin admin] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:08.763406 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acb0e51-998b-4462-ba52-a4830e424895 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:08.763743 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acb0e51-998b-4462-ba52-a4830e424895 admin admin] PATH_INFO: `/v3/users` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:08.764364 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acb0e51-998b-4462-ba52-a4830e424895 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:08.766547 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acb0e51-998b-4462-ba52-a4830e424895 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:08.775624 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c24598-036b-4223-8bfa-8e9fd8a916be admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:08.780734 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15f38b5-0489-4e9a-952d-bf85b889c3b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:08.781133 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15f38b5-0489-4e9a-952d-bf85b889c3b3 admin admin] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:08.781401 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15f38b5-0489-4e9a-952d-bf85b889c3b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:08.781759 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15f38b5-0489-4e9a-952d-bf85b889c3b3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:08.782291 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15f38b5-0489-4e9a-952d-bf85b889c3b3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:08.784932 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15f38b5-0489-4e9a-952d-bf85b889c3b3 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:08.788467 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 790/1581] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 17:40:08 2022] POST /identity/v3/users => generated 294 bytes in 41 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:40:08.789419 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acb0e51-998b-4462-ba52-a4830e424895 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:08.799981 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acb0e51-998b-4462-ba52-a4830e424895 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:08.804716 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6ad057-1c38-4836-b1ad-bc42c96915bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:08.805336 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6ad057-1c38-4836-b1ad-bc42c96915bc admin admin] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:08.805631 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6ad057-1c38-4836-b1ad-bc42c96915bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:08.806023 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6ad057-1c38-4836-b1ad-bc42c96915bc admin admin] PATH_INFO: `/v3/users` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:08.806705 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6ad057-1c38-4836-b1ad-bc42c96915bc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:08.809072 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6ad057-1c38-4836-b1ad-bc42c96915bc admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:08.818999 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 792/1582] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 17:40:08 2022] POST /identity/v3/projects => generated 383 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:40:08.818999 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15f38b5-0489-4e9a-952d-bf85b889c3b3 admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:08.821716 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:40:08.837767 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 791/1583] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 17:40:08 2022] POST /identity/v3/users => generated 297 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:40:08.837767 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6ad057-1c38-4836-b1ad-bc42c96915bc admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:08.844944 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6ad057-1c38-4836-b1ad-bc42c96915bc admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:08.870948 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:40:08.883353 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15f38b5-0489-4e9a-952d-bf85b889c3b3 admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:08.888284 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650d8b14-847e-4fda-9c6c-c7f5ef104520 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:08.888922 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650d8b14-847e-4fda-9c6c-c7f5ef104520 admin admin] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:08.889199 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650d8b14-847e-4fda-9c6c-c7f5ef104520 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:08.889469 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650d8b14-847e-4fda-9c6c-c7f5ef104520 admin admin] PATH_INFO: `/v3/projects` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:08.890092 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650d8b14-847e-4fda-9c6c-c7f5ef104520 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:08.892046 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650d8b14-847e-4fda-9c6c-c7f5ef104520 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:08.902499 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3377b08-09e6-4a68-bd41-7c6541ad2a18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:08.903209 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3377b08-09e6-4a68-bd41-7c6541ad2a18 admin admin] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:08.903485 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3377b08-09e6-4a68-bd41-7c6541ad2a18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:08.903755 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3377b08-09e6-4a68-bd41-7c6541ad2a18 admin admin] PATH_INFO: `/v3/users` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:08.904371 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3377b08-09e6-4a68-bd41-7c6541ad2a18 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:08.906365 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3377b08-09e6-4a68-bd41-7c6541ad2a18 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:08.919087 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 793/1584] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 17:40:08 2022] POST /identity/v3/projects => generated 405 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:40:08.919946 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650d8b14-847e-4fda-9c6c-c7f5ef104520 admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:08.931745 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 792/1585] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 17:40:08 2022] POST /identity/v3/users => generated 290 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:40:08.932531 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3377b08-09e6-4a68-bd41-7c6541ad2a18 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:08.941921 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3377b08-09e6-4a68-bd41-7c6541ad2a18 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:08.974867 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650d8b14-847e-4fda-9c6c-c7f5ef104520 admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:08.978594 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e27556-34a7-4bf5-8d5d-768fd5b56962 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59be342541264942ad87f60ff4de6c8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:08.979205 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e27556-34a7-4bf5-8d5d-768fd5b56962 service cinder] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:08.979484 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e27556-34a7-4bf5-8d5d-768fd5b56962 service cinder] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:08.979757 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e27556-34a7-4bf5-8d5d-768fd5b56962 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:08.999139 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b7e1c1-4ebb-4d02-a554-f287391bcc2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:08.999740 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b7e1c1-4ebb-4d02-a554-f287391bcc2e admin admin] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:09.000022 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b7e1c1-4ebb-4d02-a554-f287391bcc2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:09.000299 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b7e1c1-4ebb-4d02-a554-f287391bcc2e admin admin] PATH_INFO: `/v3/projects` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:09.000969 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b7e1c1-4ebb-4d02-a554-f287391bcc2e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:09.002973 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b7e1c1-4ebb-4d02-a554-f287391bcc2e admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:09.030221 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 793/1586] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 17:40:08 2022] POST /identity/v3/projects => generated 365 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:40:09.031057 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b7e1c1-4ebb-4d02-a554-f287391bcc2e admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:09.039543 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e27556-34a7-4bf5-8d5d-768fd5b56962 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:09.041310 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e27556-34a7-4bf5-8d5d-768fd5b56962 service cinder] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:09.051425 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 794/1587] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:40:08 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:40:09.052224 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e27556-34a7-4bf5-8d5d-768fd5b56962 service cinder] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:09.062062 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e27556-34a7-4bf5-8d5d-768fd5b56962 service cinder] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:09.104341 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b7e1c1-4ebb-4d02-a554-f287391bcc2e admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:09.109791 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f59b9a-94c2-48a6-bf2d-01f93e65adba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:09.110613 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f59b9a-94c2-48a6-bf2d-01f93e65adba admin admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:09.111528 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f59b9a-94c2-48a6-bf2d-01f93e65adba admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:09.111870 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f59b9a-94c2-48a6-bf2d-01f93e65adba admin admin] PATH_INFO: `/v3/roles` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:09.112402 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f59b9a-94c2-48a6-bf2d-01f93e65adba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:09.114081 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f59b9a-94c2-48a6-bf2d-01f93e65adba admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:09.123647 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf42e229-8a7d-4406-83a6-c5408c876ebd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:09.127476 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf42e229-8a7d-4406-83a6-c5408c876ebd admin admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:09.127608 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 794/1588] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:40:09 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:40:09.127694 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf42e229-8a7d-4406-83a6-c5408c876ebd admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:09.127694 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf42e229-8a7d-4406-83a6-c5408c876ebd admin admin] PATH_INFO: `/v3/roles` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:09.127852 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f59b9a-94c2-48a6-bf2d-01f93e65adba admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:09.127963 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf42e229-8a7d-4406-83a6-c5408c876ebd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:09.127963 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf42e229-8a7d-4406-83a6-c5408c876ebd admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:09.133949 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:40:09.140624 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 795/1589] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:40:09 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:40:09.140897 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f59b9a-94c2-48a6-bf2d-01f93e65adba admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:09.141000 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf42e229-8a7d-4406-83a6-c5408c876ebd admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:09.148152 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de718ec-9048-4ddb-bf30-87e689258ec3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:09.149039 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de718ec-9048-4ddb-bf30-87e689258ec3 admin admin] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:09.149568 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf42e229-8a7d-4406-83a6-c5408c876ebd admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:09.149700 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de718ec-9048-4ddb-bf30-87e689258ec3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:09.150179 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de718ec-9048-4ddb-bf30-87e689258ec3 admin admin] PATH_INFO: `/v3/users` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:09.150875 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de718ec-9048-4ddb-bf30-87e689258ec3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:09.153984 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de718ec-9048-4ddb-bf30-87e689258ec3 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:09.156411 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94da111-61fe-4bc5-b102-95911ced55d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:09.156999 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94da111-61fe-4bc5-b102-95911ced55d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:09.157283 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94da111-61fe-4bc5-b102-95911ced55d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:09.157610 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94da111-61fe-4bc5-b102-95911ced55d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:09.158879 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94da111-61fe-4bc5-b102-95911ced55d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:09.160481 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94da111-61fe-4bc5-b102-95911ced55d7 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:09.171316 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 796/1590] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:40:09 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:40:09.172137 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94da111-61fe-4bc5-b102-95911ced55d7 admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:09.184368 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 795/1591] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 17:40:09 2022] POST /identity/v3/users => generated 292 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:40:09.184368 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de718ec-9048-4ddb-bf30-87e689258ec3 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:09.185304 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94da111-61fe-4bc5-b102-95911ced55d7 admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:09.192222 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27121b2-62f3-4a87-b1ff-511161a18dbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:09.192876 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27121b2-62f3-4a87-b1ff-511161a18dbc admin admin] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:09.193141 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27121b2-62f3-4a87-b1ff-511161a18dbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:09.193425 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27121b2-62f3-4a87-b1ff-511161a18dbc admin admin] PATH_INFO: `/v3/users` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:09.194033 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de718ec-9048-4ddb-bf30-87e689258ec3 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:09.194154 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27121b2-62f3-4a87-b1ff-511161a18dbc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:09.195987 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27121b2-62f3-4a87-b1ff-511161a18dbc admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:09.199398 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9647372f-9e20-443a-8709-63a1e974196d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:09.200007 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9647372f-9e20-443a-8709-63a1e974196d admin admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:09.200339 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9647372f-9e20-443a-8709-63a1e974196d admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:09.200772 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9647372f-9e20-443a-8709-63a1e974196d admin admin] PATH_INFO: `/v3/roles` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:09.201421 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9647372f-9e20-443a-8709-63a1e974196d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:09.203249 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9647372f-9e20-443a-8709-63a1e974196d admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:09.214700 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:40:09.218963 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 796/1592] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:40:09 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:40:09.219968 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9647372f-9e20-443a-8709-63a1e974196d admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:09.228165 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 797/1593] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 17:40:09 2022] POST /identity/v3/users => generated 309 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:40:09.229320 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27121b2-62f3-4a87-b1ff-511161a18dbc admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:09.230884 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9647372f-9e20-443a-8709-63a1e974196d admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:09.235661 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09ec3b8-a0bc-4987-8a2d-bd47c26cd237 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:09.236270 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09ec3b8-a0bc-4987-8a2d-bd47c26cd237 admin admin] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:09.236569 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09ec3b8-a0bc-4987-8a2d-bd47c26cd237 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:09.236857 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09ec3b8-a0bc-4987-8a2d-bd47c26cd237 admin admin] PATH_INFO: `/v3/users` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:09.237513 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09ec3b8-a0bc-4987-8a2d-bd47c26cd237 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:09.239491 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09ec3b8-a0bc-4987-8a2d-bd47c26cd237 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:09.240363 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27121b2-62f3-4a87-b1ff-511161a18dbc admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:09.245282 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893e25d6-65d0-4a2e-aeb9-53b81c7d1151 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:09.245899 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893e25d6-65d0-4a2e-aeb9-53b81c7d1151 admin admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:09.246184 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893e25d6-65d0-4a2e-aeb9-53b81c7d1151 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:09.246459 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893e25d6-65d0-4a2e-aeb9-53b81c7d1151 admin admin] PATH_INFO: `/v3/roles` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:09.247101 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893e25d6-65d0-4a2e-aeb9-53b81c7d1151 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:09.248890 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893e25d6-65d0-4a2e-aeb9-53b81c7d1151 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:09.262149 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 797/1594] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 17:40:09 2022] POST /identity/v3/users => generated 283 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:40:09.263068 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09ec3b8-a0bc-4987-8a2d-bd47c26cd237 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:09.263469 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 798/1595] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:40:09 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:40:09.264444 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893e25d6-65d0-4a2e-aeb9-53b81c7d1151 admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:09.275244 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893e25d6-65d0-4a2e-aeb9-53b81c7d1151 admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:09.279667 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09ec3b8-a0bc-4987-8a2d-bd47c26cd237 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:09.280834 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3343fe86-5562-44b2-8cbd-e8963aa21c00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:09.281159 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3343fe86-5562-44b2-8cbd-e8963aa21c00 admin admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:09.281441 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3343fe86-5562-44b2-8cbd-e8963aa21c00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:09.281718 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3343fe86-5562-44b2-8cbd-e8963aa21c00 admin admin] PATH_INFO: `/v3/roles` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:09.282355 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3343fe86-5562-44b2-8cbd-e8963aa21c00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:09.284539 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3343fe86-5562-44b2-8cbd-e8963aa21c00 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:09.286788 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183e2f05-6dcf-4dbf-ab3e-209e264be189 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:09.287399 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183e2f05-6dcf-4dbf-ab3e-209e264be189 admin admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:09.287683 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183e2f05-6dcf-4dbf-ab3e-209e264be189 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:09.287962 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183e2f05-6dcf-4dbf-ab3e-209e264be189 admin admin] PATH_INFO: `/v3/roles` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:09.288648 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183e2f05-6dcf-4dbf-ab3e-209e264be189 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:09.290225 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183e2f05-6dcf-4dbf-ab3e-209e264be189 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:09.297249 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 799/1596] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:40:09 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:40:09.298636 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3343fe86-5562-44b2-8cbd-e8963aa21c00 admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:09.300987 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 798/1597] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:40:09 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:40:09.301919 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183e2f05-6dcf-4dbf-ab3e-209e264be189 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:09.311433 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3343fe86-5562-44b2-8cbd-e8963aa21c00 admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:09.314350 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183e2f05-6dcf-4dbf-ab3e-209e264be189 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:09.317137 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76ff743-c5d9-4aab-b3f6-a670e0c3a9cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:09.317793 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76ff743-c5d9-4aab-b3f6-a670e0c3a9cc admin admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:09.318051 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76ff743-c5d9-4aab-b3f6-a670e0c3a9cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:09.318324 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76ff743-c5d9-4aab-b3f6-a670e0c3a9cc admin admin] PATH_INFO: `/v3/roles` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:09.318968 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76ff743-c5d9-4aab-b3f6-a670e0c3a9cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:09.320116 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37b52a6-21c7-4ca3-a620-8aa9312a8593 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:09.320545 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76ff743-c5d9-4aab-b3f6-a670e0c3a9cc admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:09.320857 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37b52a6-21c7-4ca3-a620-8aa9312a8593 admin admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:09.321225 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37b52a6-21c7-4ca3-a620-8aa9312a8593 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:09.321568 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37b52a6-21c7-4ca3-a620-8aa9312a8593 admin admin] PATH_INFO: `/v3/roles` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:09.322334 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37b52a6-21c7-4ca3-a620-8aa9312a8593 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:09.324212 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37b52a6-21c7-4ca3-a620-8aa9312a8593 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:09.335970 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 800/1598] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:40:09 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:40:09.336395 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 799/1599] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:40:09 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:40:09.337277 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76ff743-c5d9-4aab-b3f6-a670e0c3a9cc admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:09.337405 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37b52a6-21c7-4ca3-a620-8aa9312a8593 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:09.348239 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37b52a6-21c7-4ca3-a620-8aa9312a8593 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:09.349744 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76ff743-c5d9-4aab-b3f6-a670e0c3a9cc admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:09.354581 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae47c0fb-78d4-4fa0-82c4-d82e0c48df23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:09.355193 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae47c0fb-78d4-4fa0-82c4-d82e0c48df23 admin admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:09.355479 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae47c0fb-78d4-4fa0-82c4-d82e0c48df23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:09.355616 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfd9041-f6f1-4a86-874b-fd1d97150246 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:09.355734 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae47c0fb-78d4-4fa0-82c4-d82e0c48df23 admin admin] PATH_INFO: `/v3/roles` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:09.356382 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae47c0fb-78d4-4fa0-82c4-d82e0c48df23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:09.356863 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfd9041-f6f1-4a86-874b-fd1d97150246 admin admin] REQUEST_METHOD: `PUT` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:09.357293 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfd9041-f6f1-4a86-874b-fd1d97150246 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:09.357771 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfd9041-f6f1-4a86-874b-fd1d97150246 admin admin] PATH_INFO: `/v3/projects/22d1c7b9f8a94e5abc2adbad97e5a479/users/c36a9a0d92024734b1aa007a56c6328c/roles/02476a5b82eb4b188c18a5d706a3ef68` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:09.358184 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae47c0fb-78d4-4fa0-82c4-d82e0c48df23 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:09.370826 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 801/1600] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:40:09 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:40:09.372065 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae47c0fb-78d4-4fa0-82c4-d82e0c48df23 admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:09.377395 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfd9041-f6f1-4a86-874b-fd1d97150246 admin admin] RBAC: Authorizing `identity:create_grant(project_id=22d1c7b9f8a94e5abc2adbad97e5a479, user_id=c36a9a0d92024734b1aa007a56c6328c, role_id=02476a5b82eb4b188c18a5d706a3ef68)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:09.380272 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfd9041-f6f1-4a86-874b-fd1d97150246 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:09.385166 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae47c0fb-78d4-4fa0-82c4-d82e0c48df23 admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:09.390807 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daec8e9c-bbb5-45dd-8919-4c87c20144bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:09.391535 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daec8e9c-bbb5-45dd-8919-4c87c20144bc admin admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:09.392465 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daec8e9c-bbb5-45dd-8919-4c87c20144bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:09.393032 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daec8e9c-bbb5-45dd-8919-4c87c20144bc admin admin] PATH_INFO: `/v3/roles` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:09.393747 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daec8e9c-bbb5-45dd-8919-4c87c20144bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:09.395573 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daec8e9c-bbb5-45dd-8919-4c87c20144bc admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:09.395948 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 800/1601] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 17:40:09 2022] PUT /identity/v3/projects/22d1c7b9f8a94e5abc2adbad97e5a479/users/c36a9a0d92024734b1aa007a56c6328c/roles/02476a5b82eb4b188c18a5d706a3ef68 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:40:09.397786 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfd9041-f6f1-4a86-874b-fd1d97150246 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:09.407080 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 802/1602] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:40:09 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:40:09.408027 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daec8e9c-bbb5-45dd-8919-4c87c20144bc admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:09.419456 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daec8e9c-bbb5-45dd-8919-4c87c20144bc admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:09.469554 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfd9041-f6f1-4a86-874b-fd1d97150246 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:09.475270 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffac29ec-e7f3-4681-8d37-775b26883dbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:09.476055 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffac29ec-e7f3-4681-8d37-775b26883dbc admin admin] REQUEST_METHOD: `PUT` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:09.476400 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffac29ec-e7f3-4681-8d37-775b26883dbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:09.476775 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffac29ec-e7f3-4681-8d37-775b26883dbc admin admin] PATH_INFO: `/v3/projects/c6986df7cf5d4be09aac96f460903bcc/users/40c0367417434c04948e9f2a03e880a9/roles/02476a5b82eb4b188c18a5d706a3ef68` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:09.482221 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4babf8c8-2ef3-4faf-be23-58f71f605747 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:09.482875 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4babf8c8-2ef3-4faf-be23-58f71f605747 admin admin] REQUEST_METHOD: `PUT` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:09.483159 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4babf8c8-2ef3-4faf-be23-58f71f605747 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:09.483476 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4babf8c8-2ef3-4faf-be23-58f71f605747 admin admin] PATH_INFO: `/v3/projects/761d3524ec174bcb8f207cff3728bcea/users/892dad01ea684e6cb2d5cec659c6c4b9/roles/02476a5b82eb4b188c18a5d706a3ef68` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:09.491801 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffac29ec-e7f3-4681-8d37-775b26883dbc admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6986df7cf5d4be09aac96f460903bcc, user_id=40c0367417434c04948e9f2a03e880a9, role_id=02476a5b82eb4b188c18a5d706a3ef68)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:09.494493 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffac29ec-e7f3-4681-8d37-775b26883dbc admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:09.503348 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4babf8c8-2ef3-4faf-be23-58f71f605747 admin admin] RBAC: Authorizing `identity:create_grant(project_id=761d3524ec174bcb8f207cff3728bcea, user_id=892dad01ea684e6cb2d5cec659c6c4b9, role_id=02476a5b82eb4b188c18a5d706a3ef68)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:09.507043 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4babf8c8-2ef3-4faf-be23-58f71f605747 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:09.508926 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 801/1603] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 17:40:09 2022] PUT /identity/v3/projects/c6986df7cf5d4be09aac96f460903bcc/users/40c0367417434c04948e9f2a03e880a9/roles/02476a5b82eb4b188c18a5d706a3ef68 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:40:09.509776 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffac29ec-e7f3-4681-8d37-775b26883dbc admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:09.524855 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 803/1604] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 17:40:09 2022] PUT /identity/v3/projects/761d3524ec174bcb8f207cff3728bcea/users/892dad01ea684e6cb2d5cec659c6c4b9/roles/02476a5b82eb4b188c18a5d706a3ef68 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:40:09.525593 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4babf8c8-2ef3-4faf-be23-58f71f605747 admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:09.587548 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:40:09.600569 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffac29ec-e7f3-4681-8d37-775b26883dbc admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:09.605701 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4babf8c8-2ef3-4faf-be23-58f71f605747 admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:09.608360 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004bfcb3-fe8a-4514-ba18-2ceb7ec8d937 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:09.609469 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004bfcb3-fe8a-4514-ba18-2ceb7ec8d937 admin admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:09.609925 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004bfcb3-fe8a-4514-ba18-2ceb7ec8d937 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:09.610382 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004bfcb3-fe8a-4514-ba18-2ceb7ec8d937 admin admin] PATH_INFO: `/v3/roles` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:09.610567 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276a92fe-bd50-4799-94d0-92d3e0f2482e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:09.611268 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276a92fe-bd50-4799-94d0-92d3e0f2482e admin admin] REQUEST_METHOD: `PUT` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:09.611552 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276a92fe-bd50-4799-94d0-92d3e0f2482e admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:09.611778 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004bfcb3-fe8a-4514-ba18-2ceb7ec8d937 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:09.611913 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276a92fe-bd50-4799-94d0-92d3e0f2482e admin admin] PATH_INFO: `/v3/projects/9206de8ce97b425cb5c675ec0a240d71/users/1d0ae308269b4cdd80990f7b38c0cd0c/roles/02476a5b82eb4b188c18a5d706a3ef68` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:09.614390 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004bfcb3-fe8a-4514-ba18-2ceb7ec8d937 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:09.628831 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276a92fe-bd50-4799-94d0-92d3e0f2482e admin admin] RBAC: Authorizing `identity:create_grant(project_id=9206de8ce97b425cb5c675ec0a240d71, user_id=1d0ae308269b4cdd80990f7b38c0cd0c, role_id=02476a5b82eb4b188c18a5d706a3ef68)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:09.634302 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276a92fe-bd50-4799-94d0-92d3e0f2482e admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:09.635360 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 802/1605] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:40:09 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:40:09.635360 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004bfcb3-fe8a-4514-ba18-2ceb7ec8d937 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:09.649153 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 804/1606] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 17:40:09 2022] PUT /identity/v3/projects/9206de8ce97b425cb5c675ec0a240d71/users/1d0ae308269b4cdd80990f7b38c0cd0c/roles/02476a5b82eb4b188c18a5d706a3ef68 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:40:09.650322 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004bfcb3-fe8a-4514-ba18-2ceb7ec8d937 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:09.654138 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b0255c-59ca-4e44-89f4-dfe18e8990f6 None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:09.654437 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b0255c-59ca-4e44-89f4-dfe18e8990f6 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:09.654732 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b0255c-59ca-4e44-89f4-dfe18e8990f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:09.675299 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-e2b0255c-59ca-4e44-89f4-dfe18e8990f6 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:40:09.676628 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-e2b0255c-59ca-4e44-89f4-dfe18e8990f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:09.681754 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:40:09.721402 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5ec54c-d4b6-46f8-9c9f-5d00913bf5ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:09.722064 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5ec54c-d4b6-46f8-9c9f-5d00913bf5ba admin admin] REQUEST_METHOD: `PUT` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:09.722342 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5ec54c-d4b6-46f8-9c9f-5d00913bf5ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:09.722614 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5ec54c-d4b6-46f8-9c9f-5d00913bf5ba admin admin] PATH_INFO: `/v3/projects/96e74de0a8544c2791702d69430fb507/users/29879afb511840ed88a8009b88f64637/roles/e411ad9eb96e4e8d85b59b5b700ada7b` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:09.738320 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5ec54c-d4b6-46f8-9c9f-5d00913bf5ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=96e74de0a8544c2791702d69430fb507, user_id=29879afb511840ed88a8009b88f64637, role_id=e411ad9eb96e4e8d85b59b5b700ada7b)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:09.741009 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5ec54c-d4b6-46f8-9c9f-5d00913bf5ba admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:09.745384 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 805/1607] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:40:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:40:09.746595 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b0255c-59ca-4e44-89f4-dfe18e8990f6 None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:09.759246 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 803/1608] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 17:40:09 2022] PUT /identity/v3/projects/96e74de0a8544c2791702d69430fb507/users/29879afb511840ed88a8009b88f64637/roles/e411ad9eb96e4e8d85b59b5b700ada7b => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:40:09.760794 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71cfe8f-fa11-4a95-9c77-9c98c6cb6946 None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:09.761077 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71cfe8f-fa11-4a95-9c77-9c98c6cb6946 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:09.761345 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71cfe8f-fa11-4a95-9c77-9c98c6cb6946 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:09.764249 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b0255c-59ca-4e44-89f4-dfe18e8990f6 None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:09.786360 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-a71cfe8f-fa11-4a95-9c77-9c98c6cb6946 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:40:09.787643 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-a71cfe8f-fa11-4a95-9c77-9c98c6cb6946 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:09.839699 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d020833a-0c67-4633-b7bb-5d37b7a586b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:09.840306 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d020833a-0c67-4633-b7bb-5d37b7a586b2 admin admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:09.840614 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d020833a-0c67-4633-b7bb-5d37b7a586b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:09.840883 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d020833a-0c67-4633-b7bb-5d37b7a586b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:09.841530 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d020833a-0c67-4633-b7bb-5d37b7a586b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:09.843092 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d020833a-0c67-4633-b7bb-5d37b7a586b2 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:09.852635 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 806/1609] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:40:09 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:40:09.856167 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 804/1610] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:40:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:40:09.856308 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4295360b-f484-4835-878f-ff6077ad7c71 None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:09.856393 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71cfe8f-fa11-4a95-9c77-9c98c6cb6946 None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:09.856471 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4295360b-f484-4835-878f-ff6077ad7c71 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:09.856471 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4295360b-f484-4835-878f-ff6077ad7c71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:09.865314 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71cfe8f-fa11-4a95-9c77-9c98c6cb6946 None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:09.870293 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2a292c-8601-4cd3-bef0-2ad222d8c644 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:09.870950 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2a292c-8601-4cd3-bef0-2ad222d8c644 admin admin] REQUEST_METHOD: `PUT` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:09.871228 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2a292c-8601-4cd3-bef0-2ad222d8c644 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:09.871499 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2a292c-8601-4cd3-bef0-2ad222d8c644 admin admin] PATH_INFO: `/v3/projects/3cbb522709a64ec1abdf99ede28bcae6/users/2d27ff5caebd413caf117b61e1f74c8a/roles/02476a5b82eb4b188c18a5d706a3ef68` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:09.873295 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-4295360b-f484-4835-878f-ff6077ad7c71 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:40:09.874574 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-4295360b-f484-4835-878f-ff6077ad7c71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:09.883255 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 807/1611] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:40:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:40:09.884379 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6e55a7-507f-4aab-bf5e-72f644c22ae9 None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:09.884693 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6e55a7-507f-4aab-bf5e-72f644c22ae9 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:09.885233 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6e55a7-507f-4aab-bf5e-72f644c22ae9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:09.891167 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2a292c-8601-4cd3-bef0-2ad222d8c644 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3cbb522709a64ec1abdf99ede28bcae6, user_id=2d27ff5caebd413caf117b61e1f74c8a, role_id=02476a5b82eb4b188c18a5d706a3ef68)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:09.895738 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2a292c-8601-4cd3-bef0-2ad222d8c644 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:09.905668 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-0e6e55a7-507f-4aab-bf5e-72f644c22ae9 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:40:09.906853 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-0e6e55a7-507f-4aab-bf5e-72f644c22ae9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:09.908837 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 805/1612] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 17:40:09 2022] PUT /identity/v3/projects/3cbb522709a64ec1abdf99ede28bcae6/users/2d27ff5caebd413caf117b61e1f74c8a/roles/02476a5b82eb4b188c18a5d706a3ef68 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:40:09.909654 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2a292c-8601-4cd3-bef0-2ad222d8c644 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:09.966191 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2a292c-8601-4cd3-bef0-2ad222d8c644 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:09.969867 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f281e637-c1ca-4083-97f5-3bb25f7634e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43ad98a250a144099ccd18378aedd4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:09.970192 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 808/1613] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:40:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:40:09.970501 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f281e637-c1ca-4083-97f5-3bb25f7634e7 service neutron] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:09.970781 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f281e637-c1ca-4083-97f5-3bb25f7634e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:09.971052 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f281e637-c1ca-4083-97f5-3bb25f7634e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:09.971152 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6e55a7-507f-4aab-bf5e-72f644c22ae9 None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:09.980531 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6e55a7-507f-4aab-bf5e-72f644c22ae9 None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:09.981106 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:40:09.986231 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9797e4a3-996e-43a4-9ffc-cea260d9384c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:09.986879 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9797e4a3-996e-43a4-9ffc-cea260d9384c admin admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:09.987161 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9797e4a3-996e-43a4-9ffc-cea260d9384c admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:09.987399 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f281e637-c1ca-4083-97f5-3bb25f7634e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:09.987509 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9797e4a3-996e-43a4-9ffc-cea260d9384c admin admin] PATH_INFO: `/v3/roles` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:09.988070 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9797e4a3-996e-43a4-9ffc-cea260d9384c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:09.989409 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f281e637-c1ca-4083-97f5-3bb25f7634e7 service neutron] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:09.989706 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9797e4a3-996e-43a4-9ffc-cea260d9384c admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:09.999181 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 806/1614] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:40:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:40:10.003337 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 809/1615] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:40:09 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:40:10.003479 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f281e637-c1ca-4083-97f5-3bb25f7634e7 service neutron] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:10.003627 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9797e4a3-996e-43a4-9ffc-cea260d9384c admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:10.010217 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f281e637-c1ca-4083-97f5-3bb25f7634e7 service neutron] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:10.014042 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9797e4a3-996e-43a4-9ffc-cea260d9384c admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:10.014937 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9166841-bd52-42e0-8c9a-810bb09de414 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:10.015594 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9166841-bd52-42e0-8c9a-810bb09de414 admin admin] REQUEST_METHOD: `PUT` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:10.015877 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9166841-bd52-42e0-8c9a-810bb09de414 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:10.016150 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9166841-bd52-42e0-8c9a-810bb09de414 admin admin] PATH_INFO: `/v3/projects/374189cb550944e59c52f94e702f09b8/users/7b140b246fd3488a9f271d6b496e6616/roles/02476a5b82eb4b188c18a5d706a3ef68` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:10.019114 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4caaf2ee-f519-4b88-b2b3-270ff2acef3f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43ad98a250a144099ccd18378aedd4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:10.020059 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caaf2ee-f519-4b88-b2b3-270ff2acef3f service neutron] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:10.020724 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caaf2ee-f519-4b88-b2b3-270ff2acef3f service neutron] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:10.021070 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caaf2ee-f519-4b88-b2b3-270ff2acef3f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:10.032282 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4caaf2ee-f519-4b88-b2b3-270ff2acef3f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:10.032428 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9166841-bd52-42e0-8c9a-810bb09de414 admin admin] RBAC: Authorizing `identity:create_grant(project_id=374189cb550944e59c52f94e702f09b8, user_id=7b140b246fd3488a9f271d6b496e6616, role_id=02476a5b82eb4b188c18a5d706a3ef68)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:10.033928 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4caaf2ee-f519-4b88-b2b3-270ff2acef3f service neutron] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:10.035038 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9166841-bd52-42e0-8c9a-810bb09de414 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:10.044848 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 810/1616] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:40:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:40:10.045855 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4caaf2ee-f519-4b88-b2b3-270ff2acef3f service neutron] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:10.060577 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 807/1617] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 17:40:09 2022] PUT /identity/v3/projects/374189cb550944e59c52f94e702f09b8/users/7b140b246fd3488a9f271d6b496e6616/roles/02476a5b82eb4b188c18a5d706a3ef68 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:40:10.060577 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b66c72-455d-478f-a1c2-6073775046e2 None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:10.060577 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b66c72-455d-478f-a1c2-6073775046e2 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:10.060577 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b66c72-455d-478f-a1c2-6073775046e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:10.067010 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4caaf2ee-f519-4b88-b2b3-270ff2acef3f service neutron] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:10.087589 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:40:10.098503 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-66b66c72-455d-478f-a1c2-6073775046e2 None None] MFA Rules not processed for user `2d27ff5caebd413caf117b61e1f74c8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:40:10.099969 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-66b66c72-455d-478f-a1c2-6073775046e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:10.151980 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae209ac-b174-482d-9510-05cd746dfcc2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43ad98a250a144099ccd18378aedd4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:10.152873 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae209ac-b174-482d-9510-05cd746dfcc2 service neutron] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:10.153375 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae209ac-b174-482d-9510-05cd746dfcc2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:10.153755 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae209ac-b174-482d-9510-05cd746dfcc2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:10.188867 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 808/1618] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:40:10 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:40:10.190346 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b66c72-455d-478f-a1c2-6073775046e2 None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:10.203380 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b66c72-455d-478f-a1c2-6073775046e2 None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:10.210592 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6090f8b8-49bc-4a49-ad6b-ecc08ec415b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43ad98a250a144099ccd18378aedd4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:10.211524 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6090f8b8-49bc-4a49-ad6b-ecc08ec415b1 service neutron] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:10.211924 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6090f8b8-49bc-4a49-ad6b-ecc08ec415b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:10.212335 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6090f8b8-49bc-4a49-ad6b-ecc08ec415b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:10.253581 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae209ac-b174-482d-9510-05cd746dfcc2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:10.256150 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae209ac-b174-482d-9510-05cd746dfcc2 service neutron] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:10.270823 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 811/1619] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:40:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:40:10.272050 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae209ac-b174-482d-9510-05cd746dfcc2 service neutron] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:10.285711 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae209ac-b174-482d-9510-05cd746dfcc2 service neutron] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:10.292840 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660b2318-d632-459a-884e-b3948ac81deb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:10.293732 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660b2318-d632-459a-884e-b3948ac81deb admin admin] REQUEST_METHOD: `PUT` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:10.294018 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660b2318-d632-459a-884e-b3948ac81deb admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:10.294291 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660b2318-d632-459a-884e-b3948ac81deb admin admin] PATH_INFO: `/v3/projects/96e74de0a8544c2791702d69430fb507/users/29879afb511840ed88a8009b88f64637/roles/e411ad9eb96e4e8d85b59b5b700ada7b` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:10.297844 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660b2318-d632-459a-884e-b3948ac81deb admin admin] RBAC: Authorizing `identity:create_grant(project_id=96e74de0a8544c2791702d69430fb507, user_id=29879afb511840ed88a8009b88f64637, role_id=e411ad9eb96e4e8d85b59b5b700ada7b)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:10.302488 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660b2318-d632-459a-884e-b3948ac81deb admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:10.317314 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 812/1620] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 17:40:10 2022] PUT /identity/v3/projects/96e74de0a8544c2791702d69430fb507/users/29879afb511840ed88a8009b88f64637/roles/e411ad9eb96e4e8d85b59b5b700ada7b => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:40:10.318888 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6090f8b8-49bc-4a49-ad6b-ecc08ec415b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:10.320374 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6090f8b8-49bc-4a49-ad6b-ecc08ec415b1 service neutron] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:10.327210 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378282b8-fd7c-4a26-a0f8-dd3981bb7e56 None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:10.328680 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378282b8-fd7c-4a26-a0f8-dd3981bb7e56 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:10.329064 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378282b8-fd7c-4a26-a0f8-dd3981bb7e56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:10.335274 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 809/1621] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:40:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:40:10.336804 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6dcb5a-f1a6-4818-bad2-90aac6f8e008 None None] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:10.337097 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6dcb5a-f1a6-4818-bad2-90aac6f8e008 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:10.337371 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6dcb5a-f1a6-4818-bad2-90aac6f8e008 None None] PATH_INFO: `/` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:10.343977 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 810/1622] 10.222.0.45 () {54 vars in 962 bytes} [Wed Aug 24 17:40:10 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:40:10.345900 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e4e162-7622-4a0a-9981-ead7f7151904 None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:10.346242 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e4e162-7622-4a0a-9981-ead7f7151904 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:10.346451 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e4e162-7622-4a0a-9981-ead7f7151904 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:10.371006 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-378282b8-fd7c-4a26-a0f8-dd3981bb7e56 None None] MFA Rules not processed for user `7b140b246fd3488a9f271d6b496e6616`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:40:10.372729 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-378282b8-fd7c-4a26-a0f8-dd3981bb7e56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:10.380955 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-f1e4e162-7622-4a0a-9981-ead7f7151904 None None] MFA Rules not processed for user `29879afb511840ed88a8009b88f64637`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:40:10.382958 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-f1e4e162-7622-4a0a-9981-ead7f7151904 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:10.438009 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:40:10.477635 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 811/1623] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:40:10 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:40:10.479202 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1cf07d-740f-436b-aa62-cff88d4d8f19 None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:10.480187 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1cf07d-740f-436b-aa62-cff88d4d8f19 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:10.480524 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1cf07d-740f-436b-aa62-cff88d4d8f19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:10.499958 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 813/1624] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:40:10 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:40:10.501752 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378282b8-fd7c-4a26-a0f8-dd3981bb7e56 None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:10.537086 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-6f1cf07d-740f-436b-aa62-cff88d4d8f19 None None] MFA Rules not processed for user `053b7ce6d6f649ed91a26e434dab805e`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:40:10.539574 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-6f1cf07d-740f-436b-aa62-cff88d4d8f19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:10.571412 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:40:10.608292 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378282b8-fd7c-4a26-a0f8-dd3981bb7e56 None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:10.620772 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ab945b-8318-4c05-be0a-8f5954454230 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59be342541264942ad87f60ff4de6c8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:10.621409 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ab945b-8318-4c05-be0a-8f5954454230 service cinder] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:10.621697 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ab945b-8318-4c05-be0a-8f5954454230 service cinder] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:10.622020 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ab945b-8318-4c05-be0a-8f5954454230 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:10.654670 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ab945b-8318-4c05-be0a-8f5954454230 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:10.662253 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ab945b-8318-4c05-be0a-8f5954454230 service cinder] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:10.681043 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 814/1625] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:40:10 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:40:10.682452 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ab945b-8318-4c05-be0a-8f5954454230 service cinder] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:10.682688 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 812/1626] 10.222.0.45 () {58 vars in 1059 bytes} [Wed Aug 24 17:40:10 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:40:10.692465 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1cf07d-740f-436b-aa62-cff88d4d8f19 None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:10.715371 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1cf07d-740f-436b-aa62-cff88d4d8f19 None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:10.728167 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f0e4d3-60ce-40fa-8c55-b91595aa0ecc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '053b7ce6d6f649ed91a26e434dab805e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:10.730931 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f0e4d3-60ce-40fa-8c55-b91595aa0ecc service nova] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:10.731237 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f0e4d3-60ce-40fa-8c55-b91595aa0ecc service nova] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:10.731517 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f0e4d3-60ce-40fa-8c55-b91595aa0ecc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:10.808283 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ab945b-8318-4c05-be0a-8f5954454230 service cinder] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:10.818695 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7828ec7c-3a36-487f-bd32-1f3a12061a86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:10.819520 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7828ec7c-3a36-487f-bd32-1f3a12061a86 admin admin] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:10.821143 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7828ec7c-3a36-487f-bd32-1f3a12061a86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:10.822637 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7828ec7c-3a36-487f-bd32-1f3a12061a86 admin admin] PATH_INFO: `/v3/projects` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:10.831052 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7828ec7c-3a36-487f-bd32-1f3a12061a86 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:10.831052 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7828ec7c-3a36-487f-bd32-1f3a12061a86 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:10.847288 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f0e4d3-60ce-40fa-8c55-b91595aa0ecc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:10.849470 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f0e4d3-60ce-40fa-8c55-b91595aa0ecc service nova] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:10.863942 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 813/1627] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:40:10 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:40:10.882021 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 815/1628] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 17:40:10 2022] POST /identity/v3/projects => generated 365 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:40:10.883252 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f0e4d3-60ce-40fa-8c55-b91595aa0ecc service nova] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:10.903763 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f0e4d3-60ce-40fa-8c55-b91595aa0ecc service nova] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:11.021890 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:40:11.036983 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7828ec7c-3a36-487f-bd32-1f3a12061a86 admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:11.041350 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c99d9c9-da65-4781-8039-356b485c39e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:11.043382 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c99d9c9-da65-4781-8039-356b485c39e7 admin admin] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:11.044134 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c99d9c9-da65-4781-8039-356b485c39e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:11.045098 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c99d9c9-da65-4781-8039-356b485c39e7 admin admin] PATH_INFO: `/v3/users` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:11.047872 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c99d9c9-da65-4781-8039-356b485c39e7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:11.049950 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7828ec7c-3a36-487f-bd32-1f3a12061a86 admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:11.051534 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c99d9c9-da65-4781-8039-356b485c39e7 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:11.055772 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2e959a-c589-41be-a242-994865de780f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:11.056715 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2e959a-c589-41be-a242-994865de780f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:11.057170 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2e959a-c589-41be-a242-994865de780f admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:11.057609 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2e959a-c589-41be-a242-994865de780f admin admin] PATH_INFO: `/v3/users/f10fd8e06543489eb7bd0858d8cb9f34` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:11.059002 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2e959a-c589-41be-a242-994865de780f admin admin] RBAC: Authorizing `identity:delete_user(user_id=f10fd8e06543489eb7bd0858d8cb9f34)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:11.060310 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: WARNING py.warnings [None req-5a2e959a-c589-41be-a242-994865de780f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 17:40:11.060310 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: warnings.warn(msg) Aug 24 17:40:11.060310 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: Aug 24 17:40:11.061865 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2e959a-c589-41be-a242-994865de780f admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:11.098023 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 814/1629] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 17:40:10 2022] POST /identity/v3/users => generated 289 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:40:11.102853 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c99d9c9-da65-4781-8039-356b485c39e7 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:11.123882 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c99d9c9-da65-4781-8039-356b485c39e7 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:11.136887 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14f2900-790f-4d5c-9128-98dc6ed382d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:11.138395 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14f2900-790f-4d5c-9128-98dc6ed382d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:11.139705 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14f2900-790f-4d5c-9128-98dc6ed382d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:11.139996 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14f2900-790f-4d5c-9128-98dc6ed382d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:11.143414 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14f2900-790f-4d5c-9128-98dc6ed382d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:11.145283 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14f2900-790f-4d5c-9128-98dc6ed382d7 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:11.172616 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-5a2e959a-c589-41be-a242-994865de780f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f10fd8e06543489eb7bd0858d8cb9f34'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:11.174141 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 815/1630] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:40:11 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 17:40:11.174581 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-5a2e959a-c589-41be-a242-994865de780f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f10fd8e06543489eb7bd0858d8cb9f34'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:11.177482 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14f2900-790f-4d5c-9128-98dc6ed382d7 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:11.181785 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-b14f2900-790f-4d5c-9128-98dc6ed382d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:11.196059 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-5a2e959a-c589-41be-a242-994865de780f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f10fd8e06543489eb7bd0858d8cb9f34'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:11.197123 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-5a2e959a-c589-41be-a242-994865de780f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f10fd8e06543489eb7bd0858d8cb9f34'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:11.204328 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14f2900-790f-4d5c-9128-98dc6ed382d7 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:11.228690 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-5a2e959a-c589-41be-a242-994865de780f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f10fd8e06543489eb7bd0858d8cb9f34'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:11.229799 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf39cdd-c610-43f8-84dc-eac3c8a5a211 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:11.230817 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf39cdd-c610-43f8-84dc-eac3c8a5a211 admin admin] REQUEST_METHOD: `PUT` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:11.232731 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf39cdd-c610-43f8-84dc-eac3c8a5a211 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:11.233311 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf39cdd-c610-43f8-84dc-eac3c8a5a211 admin admin] PATH_INFO: `/v3/projects/24c5a88ec45a4a33b5fc19d0f4ca45bb/users/e02c8478bfb94f4697639fe293324dbc/roles/e411ad9eb96e4e8d85b59b5b700ada7b` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:11.250516 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 816/1631] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 17:40:11 2022] DELETE /identity/v3/users/f10fd8e06543489eb7bd0858d8cb9f34 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:40:11.258067 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d43715a-a2db-4b9a-ac07-55878aef7155 None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:11.258684 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d43715a-a2db-4b9a-ac07-55878aef7155 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:11.259189 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d43715a-a2db-4b9a-ac07-55878aef7155 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:11.276541 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf39cdd-c610-43f8-84dc-eac3c8a5a211 admin admin] RBAC: Authorizing `identity:create_grant(project_id=24c5a88ec45a4a33b5fc19d0f4ca45bb, user_id=e02c8478bfb94f4697639fe293324dbc, role_id=e411ad9eb96e4e8d85b59b5b700ada7b)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:11.281839 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf39cdd-c610-43f8-84dc-eac3c8a5a211 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:11.307208 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 816/1632] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 17:40:11 2022] PUT /identity/v3/projects/24c5a88ec45a4a33b5fc19d0f4ca45bb/users/e02c8478bfb94f4697639fe293324dbc/roles/e411ad9eb96e4e8d85b59b5b700ada7b => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:40:11.310797 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf39cdd-c610-43f8-84dc-eac3c8a5a211 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:11.312982 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-0d43715a-a2db-4b9a-ac07-55878aef7155 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:40:11.317231 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-0d43715a-a2db-4b9a-ac07-55878aef7155 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:11.398437 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:40:11.410465 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf39cdd-c610-43f8-84dc-eac3c8a5a211 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:11.418049 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0b201a-4ef0-4470-8d28-87671a078631 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:11.419340 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0b201a-4ef0-4470-8d28-87671a078631 admin admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:11.420121 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0b201a-4ef0-4470-8d28-87671a078631 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:11.420888 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0b201a-4ef0-4470-8d28-87671a078631 admin admin] PATH_INFO: `/v3/roles` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:11.421764 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0b201a-4ef0-4470-8d28-87671a078631 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:11.425150 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0b201a-4ef0-4470-8d28-87671a078631 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:11.442041 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 817/1633] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:40:11 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 17:40:11.445827 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 817/1634] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:40:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:40:11.445827 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0eae8d-d34b-49df-bd52-a2a00816bc0f None None] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:11.445827 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0eae8d-d34b-49df-bd52-a2a00816bc0f None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:11.445827 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0eae8d-d34b-49df-bd52-a2a00816bc0f None None] PATH_INFO: `/` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:11.447512 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d23735f-dde7-465e-8975-6d120727054a None None] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:11.448027 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d23735f-dde7-465e-8975-6d120727054a None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:11.448502 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d23735f-dde7-465e-8975-6d120727054a None None] PATH_INFO: `/` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:11.451011 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 818/1635] 10.222.0.45 () {54 vars in 962 bytes} [Wed Aug 24 17:40:11 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:40:11.456975 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 818/1636] 10.222.0.45 () {54 vars in 912 bytes} [Wed Aug 24 17:40:11 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:40:11.457457 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d23735f-dde7-465e-8975-6d120727054a None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:11.459069 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9cf01b-b308-47f1-a43b-36ee2658412a None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:11.459355 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9cf01b-b308-47f1-a43b-36ee2658412a None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:11.459631 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9cf01b-b308-47f1-a43b-36ee2658412a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:11.475815 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d23735f-dde7-465e-8975-6d120727054a None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:11.482105 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad81185-ae59-4ae4-b32f-926b911e0875 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:11.483737 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad81185-ae59-4ae4-b32f-926b911e0875 admin admin] REQUEST_METHOD: `PUT` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:11.484403 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad81185-ae59-4ae4-b32f-926b911e0875 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:11.484939 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad81185-ae59-4ae4-b32f-926b911e0875 admin admin] PATH_INFO: `/v3/projects/24c5a88ec45a4a33b5fc19d0f4ca45bb/users/e02c8478bfb94f4697639fe293324dbc/roles/e411ad9eb96e4e8d85b59b5b700ada7b` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:11.489296 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-8c9cf01b-b308-47f1-a43b-36ee2658412a None None] MFA Rules not processed for user `053b7ce6d6f649ed91a26e434dab805e`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:40:11.489763 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad81185-ae59-4ae4-b32f-926b911e0875 admin admin] RBAC: Authorizing `identity:create_grant(project_id=24c5a88ec45a4a33b5fc19d0f4ca45bb, user_id=e02c8478bfb94f4697639fe293324dbc, role_id=e411ad9eb96e4e8d85b59b5b700ada7b)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:11.492457 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-8c9cf01b-b308-47f1-a43b-36ee2658412a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:11.495067 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad81185-ae59-4ae4-b32f-926b911e0875 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:11.509470 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 819/1637] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 17:40:11 2022] PUT /identity/v3/projects/24c5a88ec45a4a33b5fc19d0f4ca45bb/users/e02c8478bfb94f4697639fe293324dbc/roles/e411ad9eb96e4e8d85b59b5b700ada7b => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:40:11.511646 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853cad7e-db3e-4fd1-b3a7-407189f4a352 None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:11.512139 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853cad7e-db3e-4fd1-b3a7-407189f4a352 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:11.512688 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853cad7e-db3e-4fd1-b3a7-407189f4a352 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:11.546771 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-853cad7e-db3e-4fd1-b3a7-407189f4a352 None None] MFA Rules not processed for user `053b7ce6d6f649ed91a26e434dab805e`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:40:11.550575 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-853cad7e-db3e-4fd1-b3a7-407189f4a352 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:11.607786 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 819/1638] 10.222.0.45 () {58 vars in 1009 bytes} [Wed Aug 24 17:40:11 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:40:11.609214 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9cf01b-b308-47f1-a43b-36ee2658412a None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:11.612076 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-8c9cf01b-b308-47f1-a43b-36ee2658412a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:11.663491 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 820/1639] 10.222.0.45 () {58 vars in 1059 bytes} [Wed Aug 24 17:40:11 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:40:11.665644 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f0e059-7254-468e-a01c-1f9326cc32c3 None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:11.667269 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f0e059-7254-468e-a01c-1f9326cc32c3 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:11.671086 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f0e059-7254-468e-a01c-1f9326cc32c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:11.726255 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-45f0e059-7254-468e-a01c-1f9326cc32c3 None None] MFA Rules not processed for user `e02c8478bfb94f4697639fe293324dbc`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:40:11.727017 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9cf01b-b308-47f1-a43b-36ee2658412a None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:11.728371 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-45f0e059-7254-468e-a01c-1f9326cc32c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:11.748336 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:40:11.834030 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f53d54a-2766-4994-93fd-363880b9b4eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43ad98a250a144099ccd18378aedd4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:11.834665 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f53d54a-2766-4994-93fd-363880b9b4eb service neutron] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:11.835397 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f53d54a-2766-4994-93fd-363880b9b4eb service neutron] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:11.835684 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f53d54a-2766-4994-93fd-363880b9b4eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:11.888804 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 821/1640] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:40:11 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:40:11.905530 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f0e059-7254-468e-a01c-1f9326cc32c3 None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:11.907179 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-45f0e059-7254-468e-a01c-1f9326cc32c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:11.910722 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:40:11.951792 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f53d54a-2766-4994-93fd-363880b9b4eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:11.953247 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f53d54a-2766-4994-93fd-363880b9b4eb service neutron] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:11.986540 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 820/1641] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:40:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:40:11.989475 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad04d5b8-e64f-4f45-a4c9-90aa8c0ea2e8 None None] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:11.989762 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad04d5b8-e64f-4f45-a4c9-90aa8c0ea2e8 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:11.990040 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad04d5b8-e64f-4f45-a4c9-90aa8c0ea2e8 None None] PATH_INFO: `/` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:11.991552 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 821/1642] 10.222.0.45 () {54 vars in 912 bytes} [Wed Aug 24 17:40:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:40:11.994250 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad04d5b8-e64f-4f45-a4c9-90aa8c0ea2e8 None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:12.000206 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-ad04d5b8-e64f-4f45-a4c9-90aa8c0ea2e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:12.091737 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f0e059-7254-468e-a01c-1f9326cc32c3 None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:12.101182 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0082a328-5588-4c09-8861-e843c4b7f593 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59be342541264942ad87f60ff4de6c8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:12.103982 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0082a328-5588-4c09-8861-e843c4b7f593 service cinder] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:12.106414 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0082a328-5588-4c09-8861-e843c4b7f593 service cinder] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:12.106883 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0082a328-5588-4c09-8861-e843c4b7f593 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:12.141558 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0082a328-5588-4c09-8861-e843c4b7f593 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:12.144311 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0082a328-5588-4c09-8861-e843c4b7f593 service cinder] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:12.187853 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 822/1643] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:40:11 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:40:12.194219 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0694ef34-3f42-4e67-a7d2-d5b41cc57600 None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:12.194538 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0694ef34-3f42-4e67-a7d2-d5b41cc57600 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:12.196272 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0694ef34-3f42-4e67-a7d2-d5b41cc57600 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:12.203297 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad04d5b8-e64f-4f45-a4c9-90aa8c0ea2e8 None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:12.211805 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f13c11-218e-49cd-93da-6dd02f480aeb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59be342541264942ad87f60ff4de6c8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:12.212442 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f13c11-218e-49cd-93da-6dd02f480aeb service cinder] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:12.212752 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f13c11-218e-49cd-93da-6dd02f480aeb service cinder] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:12.213028 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f13c11-218e-49cd-93da-6dd02f480aeb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:12.247853 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f13c11-218e-49cd-93da-6dd02f480aeb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:12.252434 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-0694ef34-3f42-4e67-a7d2-d5b41cc57600 None None] MFA Rules not processed for user `053b7ce6d6f649ed91a26e434dab805e`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:40:12.253723 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-0694ef34-3f42-4e67-a7d2-d5b41cc57600 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:12.255224 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f13c11-218e-49cd-93da-6dd02f480aeb service cinder] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:12.277732 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 823/1644] 10.222.0.45 () {58 vars in 1009 bytes} [Wed Aug 24 17:40:12 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:40:12.278250 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 822/1645] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:40:11 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 286 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:40:12.389202 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0694ef34-3f42-4e67-a7d2-d5b41cc57600 None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:12.402615 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0694ef34-3f42-4e67-a7d2-d5b41cc57600 None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:12.412021 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d4fcb2-a953-4cf6-8ef3-6c0f5f7f626a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59be342541264942ad87f60ff4de6c8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:12.414862 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d4fcb2-a953-4cf6-8ef3-6c0f5f7f626a service cinder] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:12.415170 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d4fcb2-a953-4cf6-8ef3-6c0f5f7f626a service cinder] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:12.415456 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d4fcb2-a953-4cf6-8ef3-6c0f5f7f626a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:12.444182 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d4fcb2-a953-4cf6-8ef3-6c0f5f7f626a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:12.446196 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d4fcb2-a953-4cf6-8ef3-6c0f5f7f626a service cinder] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:12.482579 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 824/1646] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:40:12 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:40:13.380904 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f13c11-218e-49cd-93da-6dd02f480aeb service cinder] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:13.382605 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-02f13c11-218e-49cd-93da-6dd02f480aeb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:13.404177 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f13c11-218e-49cd-93da-6dd02f480aeb service cinder] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:13.427371 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e84a3ac-f606-4edf-b391-0e1ca12f7f9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:13.428140 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e84a3ac-f606-4edf-b391-0e1ca12f7f9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:13.428475 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e84a3ac-f606-4edf-b391-0e1ca12f7f9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:13.429538 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e84a3ac-f606-4edf-b391-0e1ca12f7f9f admin admin] PATH_INFO: `/v3/projects/acc9a400b1364f3eb1187796992fa846` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:13.451555 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e84a3ac-f606-4edf-b391-0e1ca12f7f9f admin admin] RBAC: Authorizing `identity:delete_project(project_id=acc9a400b1364f3eb1187796992fa846)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:13.458443 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e84a3ac-f606-4edf-b391-0e1ca12f7f9f admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:13.610514 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-1e84a3ac-f606-4edf-b391-0e1ca12f7f9f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'acc9a400b1364f3eb1187796992fa846', 'request_id': 'req-1e84a3ac-f606-4edf-b391-0e1ca12f7f9f'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:13.628135 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-1e84a3ac-f606-4edf-b391-0e1ca12f7f9f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'acc9a400b1364f3eb1187796992fa846', 'request_id': 'req-1e84a3ac-f606-4edf-b391-0e1ca12f7f9f'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:13.643138 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-1e84a3ac-f606-4edf-b391-0e1ca12f7f9f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'acc9a400b1364f3eb1187796992fa846', 'request_id': 'req-1e84a3ac-f606-4edf-b391-0e1ca12f7f9f'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:13.663558 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-1e84a3ac-f606-4edf-b391-0e1ca12f7f9f admin admin] The token cache is being invalidate because project acc9a400b1364f3eb1187796992fa846 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81758) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:40:13.664065 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-1e84a3ac-f606-4edf-b391-0e1ca12f7f9f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:13.664887 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-1e84a3ac-f606-4edf-b391-0e1ca12f7f9f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:13.670494 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 823/1647] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 17:40:13 2022] DELETE /identity/v3/projects/acc9a400b1364f3eb1187796992fa846 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:40:13.671287 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d4fcb2-a953-4cf6-8ef3-6c0f5f7f626a service cinder] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:13.685337 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d4fcb2-a953-4cf6-8ef3-6c0f5f7f626a service cinder] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:13.806065 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe2ff0d-f8e4-4302-a1ce-f99f4eda2ce3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:13.806833 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe2ff0d-f8e4-4302-a1ce-f99f4eda2ce3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:13.807167 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe2ff0d-f8e4-4302-a1ce-f99f4eda2ce3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:13.807507 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe2ff0d-f8e4-4302-a1ce-f99f4eda2ce3 admin admin] PATH_INFO: `/v3/users/29879afb511840ed88a8009b88f64637` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:13.808779 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe2ff0d-f8e4-4302-a1ce-f99f4eda2ce3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=29879afb511840ed88a8009b88f64637)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:13.811787 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe2ff0d-f8e4-4302-a1ce-f99f4eda2ce3 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:13.875950 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:40:13.895827 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-cfe2ff0d-f8e4-4302-a1ce-f99f4eda2ce3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '29879afb511840ed88a8009b88f64637'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:13.896496 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-cfe2ff0d-f8e4-4302-a1ce-f99f4eda2ce3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '29879afb511840ed88a8009b88f64637'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:13.907051 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-cfe2ff0d-f8e4-4302-a1ce-f99f4eda2ce3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '29879afb511840ed88a8009b88f64637'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:13.908610 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-cfe2ff0d-f8e4-4302-a1ce-f99f4eda2ce3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '29879afb511840ed88a8009b88f64637'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:13.935850 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-cfe2ff0d-f8e4-4302-a1ce-f99f4eda2ce3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '29879afb511840ed88a8009b88f64637'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:13.961597 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 825/1648] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 17:40:13 2022] DELETE /identity/v3/users/29879afb511840ed88a8009b88f64637 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:40:14.030861 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e84a3ac-f606-4edf-b391-0e1ca12f7f9f admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:14.033660 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-1e84a3ac-f606-4edf-b391-0e1ca12f7f9f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:14.051131 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e84a3ac-f606-4edf-b391-0e1ca12f7f9f admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:14.151505 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdb7b8f-228b-408a-a1a4-6131a91997cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:14.152444 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdb7b8f-228b-408a-a1a4-6131a91997cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:14.152907 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdb7b8f-228b-408a-a1a4-6131a91997cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:14.153286 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdb7b8f-228b-408a-a1a4-6131a91997cc admin admin] PATH_INFO: `/v3/users/7b140b246fd3488a9f271d6b496e6616` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:14.159773 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdb7b8f-228b-408a-a1a4-6131a91997cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=7b140b246fd3488a9f271d6b496e6616)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:14.162688 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdb7b8f-228b-408a-a1a4-6131a91997cc admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:14.169525 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:40:14.258308 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-bfdb7b8f-228b-408a-a1a4-6131a91997cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b140b246fd3488a9f271d6b496e6616'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:14.260297 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-bfdb7b8f-228b-408a-a1a4-6131a91997cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b140b246fd3488a9f271d6b496e6616'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:14.275217 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-bfdb7b8f-228b-408a-a1a4-6131a91997cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b140b246fd3488a9f271d6b496e6616'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:14.276099 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-bfdb7b8f-228b-408a-a1a4-6131a91997cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b140b246fd3488a9f271d6b496e6616'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:14.324933 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-bfdb7b8f-228b-408a-a1a4-6131a91997cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b140b246fd3488a9f271d6b496e6616'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:14.365083 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 824/1649] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 17:40:14 2022] DELETE /identity/v3/users/7b140b246fd3488a9f271d6b496e6616 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:40:14.369402 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed3a217-b24e-4faa-a704-94b398e57f9a None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:14.369702 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed3a217-b24e-4faa-a704-94b398e57f9a None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:14.369981 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed3a217-b24e-4faa-a704-94b398e57f9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:14.407064 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-8ed3a217-b24e-4faa-a704-94b398e57f9a None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:40:14.408388 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-8ed3a217-b24e-4faa-a704-94b398e57f9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:14.517201 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 826/1650] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:40:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:40:14.531759 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdb7b8f-228b-408a-a1a4-6131a91997cc admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:14.533409 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-bfdb7b8f-228b-408a-a1a4-6131a91997cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:14.590330 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed3a217-b24e-4faa-a704-94b398e57f9a None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:14.592517 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-8ed3a217-b24e-4faa-a704-94b398e57f9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:14.616610 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed3a217-b24e-4faa-a704-94b398e57f9a None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:14.628938 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a93522-8b6e-4d71-a04d-711a8e0173de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:14.631100 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a93522-8b6e-4d71-a04d-711a8e0173de admin admin] REQUEST_METHOD: `DELETE` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:14.631677 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a93522-8b6e-4d71-a04d-711a8e0173de admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:14.632099 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a93522-8b6e-4d71-a04d-711a8e0173de admin admin] PATH_INFO: `/v3/projects/96e74de0a8544c2791702d69430fb507` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:14.634818 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a93522-8b6e-4d71-a04d-711a8e0173de admin admin] RBAC: Authorizing `identity:delete_project(project_id=96e74de0a8544c2791702d69430fb507)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:14.636375 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: WARNING py.warnings [None req-01a93522-8b6e-4d71-a04d-711a8e0173de admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 17:40:14.636375 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: warnings.warn(msg) Aug 24 17:40:14.636375 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: Aug 24 17:40:14.638821 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a93522-8b6e-4d71-a04d-711a8e0173de admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:14.713821 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdb7b8f-228b-408a-a1a4-6131a91997cc admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:14.725793 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bf13e2-18c5-484b-96b6-7b65294b99fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43ad98a250a144099ccd18378aedd4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:14.726702 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bf13e2-18c5-484b-96b6-7b65294b99fa service neutron] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:14.727164 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bf13e2-18c5-484b-96b6-7b65294b99fa service neutron] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:14.727610 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bf13e2-18c5-484b-96b6-7b65294b99fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:14.751931 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bf13e2-18c5-484b-96b6-7b65294b99fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:14.754240 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bf13e2-18c5-484b-96b6-7b65294b99fa service neutron] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:14.772215 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 825/1651] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:40:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:40:14.792138 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-01a93522-8b6e-4d71-a04d-711a8e0173de admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '96e74de0a8544c2791702d69430fb507', 'request_id': 'req-01a93522-8b6e-4d71-a04d-711a8e0173de'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:14.816580 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-01a93522-8b6e-4d71-a04d-711a8e0173de admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '96e74de0a8544c2791702d69430fb507', 'request_id': 'req-01a93522-8b6e-4d71-a04d-711a8e0173de'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:14.850380 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-01a93522-8b6e-4d71-a04d-711a8e0173de admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '96e74de0a8544c2791702d69430fb507', 'request_id': 'req-01a93522-8b6e-4d71-a04d-711a8e0173de'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:14.877950 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-01a93522-8b6e-4d71-a04d-711a8e0173de admin admin] The token cache is being invalidate because project 96e74de0a8544c2791702d69430fb507 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81759) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:40:14.878447 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-01a93522-8b6e-4d71-a04d-711a8e0173de admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:14.879747 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-01a93522-8b6e-4d71-a04d-711a8e0173de admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:14.887229 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 827/1652] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 17:40:14 2022] DELETE /identity/v3/projects/96e74de0a8544c2791702d69430fb507 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:40:14.887962 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16064971-38cc-4fb5-a651-35d93f9f2713 None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:14.888256 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16064971-38cc-4fb5-a651-35d93f9f2713 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:14.888536 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16064971-38cc-4fb5-a651-35d93f9f2713 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:14.989097 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-16064971-38cc-4fb5-a651-35d93f9f2713 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:40:14.990348 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-16064971-38cc-4fb5-a651-35d93f9f2713 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:15.191137 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 826/1653] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:40:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 299 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:40:15.194795 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41bbe97-5083-4522-9e33-77dcbd1bda1e None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:15.195295 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41bbe97-5083-4522-9e33-77dcbd1bda1e None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:15.195857 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41bbe97-5083-4522-9e33-77dcbd1bda1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:15.235810 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-d41bbe97-5083-4522-9e33-77dcbd1bda1e None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:40:15.237570 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-d41bbe97-5083-4522-9e33-77dcbd1bda1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:15.252063 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 828/1654] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:40:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:40:15.252733 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16064971-38cc-4fb5-a651-35d93f9f2713 None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:15.272650 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16064971-38cc-4fb5-a651-35d93f9f2713 None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:15.279578 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f68634-1af4-42e4-9b10-38a4120736b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:15.280389 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f68634-1af4-42e4-9b10-38a4120736b3 admin admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:15.280810 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f68634-1af4-42e4-9b10-38a4120736b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:15.281421 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f68634-1af4-42e4-9b10-38a4120736b3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:15.282197 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f68634-1af4-42e4-9b10-38a4120736b3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:40:15.283335 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f68634-1af4-42e4-9b10-38a4120736b3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:15.287226 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f68634-1af4-42e4-9b10-38a4120736b3 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:15.310322 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 827/1655] 10.222.0.45 () {58 vars in 1153 bytes} [Wed Aug 24 17:40:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:40:15.315256 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41bbe97-5083-4522-9e33-77dcbd1bda1e None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:15.331888 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41bbe97-5083-4522-9e33-77dcbd1bda1e None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:15.346098 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6ab536-d5dc-41ad-b4a9-0557d3da2c5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:15.346914 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6ab536-d5dc-41ad-b4a9-0557d3da2c5b admin admin] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:15.349460 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6ab536-d5dc-41ad-b4a9-0557d3da2c5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:15.349881 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6ab536-d5dc-41ad-b4a9-0557d3da2c5b admin admin] PATH_INFO: `/v3/projects` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:15.352009 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6ab536-d5dc-41ad-b4a9-0557d3da2c5b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:15.354159 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6ab536-d5dc-41ad-b4a9-0557d3da2c5b admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:15.437991 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 829/1656] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 17:40:15 2022] POST /identity/v3/projects => generated 375 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:40:15.439161 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f68634-1af4-42e4-9b10-38a4120736b3 admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:15.460294 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f68634-1af4-42e4-9b10-38a4120736b3 admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:15.540900 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:40:15.695569 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579d38eb-1cf3-41e3-9428-61ec9f9f64bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:15.696289 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579d38eb-1cf3-41e3-9428-61ec9f9f64bf admin admin] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:15.696780 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579d38eb-1cf3-41e3-9428-61ec9f9f64bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:15.697463 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579d38eb-1cf3-41e3-9428-61ec9f9f64bf admin admin] PATH_INFO: `/v3/users` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:15.709655 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579d38eb-1cf3-41e3-9428-61ec9f9f64bf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:15.713531 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579d38eb-1cf3-41e3-9428-61ec9f9f64bf admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:15.771018 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6ab536-d5dc-41ad-b4a9-0557d3da2c5b admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:15.771564 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 828/1657] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 17:40:15 2022] POST /identity/v3/users => generated 288 bytes in 331 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:40:15.808491 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6ab536-d5dc-41ad-b4a9-0557d3da2c5b admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:15.873197 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b7c8c7-e709-4af8-ac86-0e6a46639cc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:15.877754 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b7c8c7-e709-4af8-ac86-0e6a46639cc1 admin admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:15.880066 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b7c8c7-e709-4af8-ac86-0e6a46639cc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:15.881235 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b7c8c7-e709-4af8-ac86-0e6a46639cc1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:15.882221 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b7c8c7-e709-4af8-ac86-0e6a46639cc1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:15.888830 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b7c8c7-e709-4af8-ac86-0e6a46639cc1 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:15.898394 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:40:15.936002 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579d38eb-1cf3-41e3-9428-61ec9f9f64bf admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:15.946644 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 830/1658] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:40:15 2022] GET /identity/v3/roles => generated 1420 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:40:15.951323 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579d38eb-1cf3-41e3-9428-61ec9f9f64bf admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:15.978545 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3758cef1-cf91-427f-b0f9-89a56f1c7d63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:15.984665 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3758cef1-cf91-427f-b0f9-89a56f1c7d63 admin admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:15.985107 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3758cef1-cf91-427f-b0f9-89a56f1c7d63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:15.985494 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3758cef1-cf91-427f-b0f9-89a56f1c7d63 admin admin] PATH_INFO: `/v3/roles` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:15.987092 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3758cef1-cf91-427f-b0f9-89a56f1c7d63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:15.991413 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3758cef1-cf91-427f-b0f9-89a56f1c7d63 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:16.018788 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 829/1659] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:40:15 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 17:40:16.022827 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b7c8c7-e709-4af8-ac86-0e6a46639cc1 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:16.052830 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b7c8c7-e709-4af8-ac86-0e6a46639cc1 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:16.060125 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc0c6cd-3a98-41ef-9b21-3d342784a3c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:16.076709 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc0c6cd-3a98-41ef-9b21-3d342784a3c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:16.077639 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc0c6cd-3a98-41ef-9b21-3d342784a3c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:16.077639 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc0c6cd-3a98-41ef-9b21-3d342784a3c3 admin admin] PATH_INFO: `/v3/projects/13887aec17eb4598a8535dd7f5599d39/users/deaca783962e4cedb863f5bba76a4344/roles/02476a5b82eb4b188c18a5d706a3ef68` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:16.120378 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc0c6cd-3a98-41ef-9b21-3d342784a3c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=13887aec17eb4598a8535dd7f5599d39, user_id=deaca783962e4cedb863f5bba76a4344, role_id=02476a5b82eb4b188c18a5d706a3ef68)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:16.124223 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc0c6cd-3a98-41ef-9b21-3d342784a3c3 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:16.157026 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 831/1660] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 17:40:16 2022] PUT /identity/v3/projects/13887aec17eb4598a8535dd7f5599d39/users/deaca783962e4cedb863f5bba76a4344/roles/02476a5b82eb4b188c18a5d706a3ef68 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:40:16.163562 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1cd38b-84d7-41ef-a74d-0d56b3357b1e None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:16.164311 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1cd38b-84d7-41ef-a74d-0d56b3357b1e None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:16.164846 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1cd38b-84d7-41ef-a74d-0d56b3357b1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:16.272882 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-eb1cd38b-84d7-41ef-a74d-0d56b3357b1e None None] MFA Rules not processed for user `deaca783962e4cedb863f5bba76a4344`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:40:16.284181 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-eb1cd38b-84d7-41ef-a74d-0d56b3357b1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:16.466552 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 830/1661] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:40:16 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 304 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:40:16.487655 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc0c6cd-3a98-41ef-9b21-3d342784a3c3 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:16.679639 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1cd38b-84d7-41ef-a74d-0d56b3357b1e None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:16.684710 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc0c6cd-3a98-41ef-9b21-3d342784a3c3 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:16.684840 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-eb1cd38b-84d7-41ef-a74d-0d56b3357b1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:16.708917 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1cd38b-84d7-41ef-a74d-0d56b3357b1e None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:16.721548 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429afd60-6286-4651-8c3b-5360c1bc7610 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:16.722194 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429afd60-6286-4651-8c3b-5360c1bc7610 admin admin] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:16.722477 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429afd60-6286-4651-8c3b-5360c1bc7610 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:16.722747 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429afd60-6286-4651-8c3b-5360c1bc7610 admin admin] PATH_INFO: `/v3/projects` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:16.723383 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429afd60-6286-4651-8c3b-5360c1bc7610 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:16.725782 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429afd60-6286-4651-8c3b-5360c1bc7610 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:16.726165 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0868ce09-1386-4f32-a025-a2be0a66a0b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:16.726834 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0868ce09-1386-4f32-a025-a2be0a66a0b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:16.727104 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0868ce09-1386-4f32-a025-a2be0a66a0b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:16.727372 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0868ce09-1386-4f32-a025-a2be0a66a0b7 admin admin] PATH_INFO: `/v3/projects/374189cb550944e59c52f94e702f09b8` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:16.734678 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0868ce09-1386-4f32-a025-a2be0a66a0b7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=374189cb550944e59c52f94e702f09b8)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:16.736699 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0868ce09-1386-4f32-a025-a2be0a66a0b7 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:16.788293 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 832/1662] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 17:40:16 2022] POST /identity/v3/projects => generated 377 bytes in 301 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:40:16.791725 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429afd60-6286-4651-8c3b-5360c1bc7610 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:16.854568 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-0868ce09-1386-4f32-a025-a2be0a66a0b7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '374189cb550944e59c52f94e702f09b8', 'request_id': 'req-0868ce09-1386-4f32-a025-a2be0a66a0b7'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:16.876944 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-0868ce09-1386-4f32-a025-a2be0a66a0b7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '374189cb550944e59c52f94e702f09b8', 'request_id': 'req-0868ce09-1386-4f32-a025-a2be0a66a0b7'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:16.894340 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-0868ce09-1386-4f32-a025-a2be0a66a0b7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '374189cb550944e59c52f94e702f09b8', 'request_id': 'req-0868ce09-1386-4f32-a025-a2be0a66a0b7'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:16.908695 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-0868ce09-1386-4f32-a025-a2be0a66a0b7 admin admin] The token cache is being invalidate because project 374189cb550944e59c52f94e702f09b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81758) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:40:16.909422 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-0868ce09-1386-4f32-a025-a2be0a66a0b7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:16.910544 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-0868ce09-1386-4f32-a025-a2be0a66a0b7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:16.915680 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 831/1663] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 17:40:16 2022] DELETE /identity/v3/projects/374189cb550944e59c52f94e702f09b8 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:40:16.916965 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0868ce09-1386-4f32-a025-a2be0a66a0b7 admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:16.918755 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-0868ce09-1386-4f32-a025-a2be0a66a0b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:16.943048 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:40:16.978191 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429afd60-6286-4651-8c3b-5360c1bc7610 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:17.155940 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0868ce09-1386-4f32-a025-a2be0a66a0b7 admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:17.160742 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2f9bc4-67d4-4375-8c1f-1d4a6595c18d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:17.161418 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2f9bc4-67d4-4375-8c1f-1d4a6595c18d admin admin] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:17.161689 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2f9bc4-67d4-4375-8c1f-1d4a6595c18d admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:17.161963 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2f9bc4-67d4-4375-8c1f-1d4a6595c18d admin admin] PATH_INFO: `/v3/users` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:17.162605 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2f9bc4-67d4-4375-8c1f-1d4a6595c18d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:17.164770 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2f9bc4-67d4-4375-8c1f-1d4a6595c18d admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:17.180055 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a9db02-4981-4184-bcbc-c8e2cb9be2b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:17.181015 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a9db02-4981-4184-bcbc-c8e2cb9be2b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:17.182715 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a9db02-4981-4184-bcbc-c8e2cb9be2b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:17.183775 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a9db02-4981-4184-bcbc-c8e2cb9be2b9 admin admin] PATH_INFO: `/v3/users/e02c8478bfb94f4697639fe293324dbc` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:17.187818 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a9db02-4981-4184-bcbc-c8e2cb9be2b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e02c8478bfb94f4697639fe293324dbc)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:17.192941 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a9db02-4981-4184-bcbc-c8e2cb9be2b9 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:17.209826 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:40:17.257017 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 833/1664] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 17:40:16 2022] POST /identity/v3/users => generated 295 bytes in 466 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:40:17.265813 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2f9bc4-67d4-4375-8c1f-1d4a6595c18d admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:17.269627 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-1f2f9bc4-67d4-4375-8c1f-1d4a6595c18d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:17.320038 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2f9bc4-67d4-4375-8c1f-1d4a6595c18d admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:17.338463 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3571cd6-0193-4298-a47e-d5791115fb66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:17.339098 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3571cd6-0193-4298-a47e-d5791115fb66 admin admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:17.341800 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3571cd6-0193-4298-a47e-d5791115fb66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:17.342125 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3571cd6-0193-4298-a47e-d5791115fb66 admin admin] PATH_INFO: `/v3/roles` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:17.342828 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3571cd6-0193-4298-a47e-d5791115fb66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:17.356761 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3571cd6-0193-4298-a47e-d5791115fb66 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:17.365886 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-34a9db02-4981-4184-bcbc-c8e2cb9be2b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e02c8478bfb94f4697639fe293324dbc'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:17.368316 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-34a9db02-4981-4184-bcbc-c8e2cb9be2b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e02c8478bfb94f4697639fe293324dbc'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:17.386247 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-34a9db02-4981-4184-bcbc-c8e2cb9be2b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e02c8478bfb94f4697639fe293324dbc'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:17.387341 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-34a9db02-4981-4184-bcbc-c8e2cb9be2b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e02c8478bfb94f4697639fe293324dbc'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:17.404516 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-34a9db02-4981-4184-bcbc-c8e2cb9be2b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e02c8478bfb94f4697639fe293324dbc'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:17.405579 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 834/1665] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:40:17 2022] GET /identity/v3/roles => generated 1420 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:40:17.413120 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3571cd6-0193-4298-a47e-d5791115fb66 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:17.415521 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-b3571cd6-0193-4298-a47e-d5791115fb66 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:17.441434 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3571cd6-0193-4298-a47e-d5791115fb66 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:17.446773 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991e0b62-8005-42c2-8149-638c8e27d750 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:17.447388 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991e0b62-8005-42c2-8149-638c8e27d750 admin admin] REQUEST_METHOD: `PUT` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:17.447662 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991e0b62-8005-42c2-8149-638c8e27d750 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:17.447940 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991e0b62-8005-42c2-8149-638c8e27d750 admin admin] PATH_INFO: `/v3/projects/d834ee64b8bb450c9ecd872b5c194862/users/3f43ce59bbf94d189ca0ddc555e22bf3/roles/e411ad9eb96e4e8d85b59b5b700ada7b` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:17.465766 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 832/1666] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 17:40:16 2022] DELETE /identity/v3/users/e02c8478bfb94f4697639fe293324dbc => generated 0 bytes in 549 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:40:17.471577 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991e0b62-8005-42c2-8149-638c8e27d750 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d834ee64b8bb450c9ecd872b5c194862, user_id=3f43ce59bbf94d189ca0ddc555e22bf3, role_id=e411ad9eb96e4e8d85b59b5b700ada7b)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:17.474743 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991e0b62-8005-42c2-8149-638c8e27d750 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:17.499228 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 835/1667] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 17:40:17 2022] PUT /identity/v3/projects/d834ee64b8bb450c9ecd872b5c194862/users/3f43ce59bbf94d189ca0ddc555e22bf3/roles/e411ad9eb96e4e8d85b59b5b700ada7b => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:40:17.503381 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a9db02-4981-4184-bcbc-c8e2cb9be2b9 admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:17.653047 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a9db02-4981-4184-bcbc-c8e2cb9be2b9 admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:17.734065 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:40:17.816390 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98131563-f0ef-43e0-8ac8-40831580f6c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:17.817251 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98131563-f0ef-43e0-8ac8-40831580f6c2 admin admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:17.817546 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98131563-f0ef-43e0-8ac8-40831580f6c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:17.817836 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98131563-f0ef-43e0-8ac8-40831580f6c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:17.819349 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98131563-f0ef-43e0-8ac8-40831580f6c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:17.821225 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98131563-f0ef-43e0-8ac8-40831580f6c2 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:17.837172 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 833/1668] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:40:17 2022] GET /identity/v3/roles => generated 1420 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:40:17.841212 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991e0b62-8005-42c2-8149-638c8e27d750 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:17.855874 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991e0b62-8005-42c2-8149-638c8e27d750 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:17.868344 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22aed6a6-71c6-4c97-b2c2-8a36695848f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:17.869338 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22aed6a6-71c6-4c97-b2c2-8a36695848f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:17.869623 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22aed6a6-71c6-4c97-b2c2-8a36695848f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:17.869902 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22aed6a6-71c6-4c97-b2c2-8a36695848f2 admin admin] PATH_INFO: `/v3/projects/d834ee64b8bb450c9ecd872b5c194862/users/3f43ce59bbf94d189ca0ddc555e22bf3/roles/e411ad9eb96e4e8d85b59b5b700ada7b` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:17.876019 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22aed6a6-71c6-4c97-b2c2-8a36695848f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d834ee64b8bb450c9ecd872b5c194862, user_id=3f43ce59bbf94d189ca0ddc555e22bf3, role_id=e411ad9eb96e4e8d85b59b5b700ada7b)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:17.879310 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22aed6a6-71c6-4c97-b2c2-8a36695848f2 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:17.907079 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 836/1669] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 17:40:17 2022] PUT /identity/v3/projects/d834ee64b8bb450c9ecd872b5c194862/users/3f43ce59bbf94d189ca0ddc555e22bf3/roles/e411ad9eb96e4e8d85b59b5b700ada7b => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:40:17.917261 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2bff43-f50f-4698-b19a-8a22db5a042f None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:17.919001 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2bff43-f50f-4698-b19a-8a22db5a042f None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:17.920050 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2bff43-f50f-4698-b19a-8a22db5a042f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:17.971139 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-4a2bff43-f50f-4698-b19a-8a22db5a042f None None] MFA Rules not processed for user `3f43ce59bbf94d189ca0ddc555e22bf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:40:17.972506 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-4a2bff43-f50f-4698-b19a-8a22db5a042f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:18.056753 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 834/1670] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:40:17 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:40:18.068397 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22aed6a6-71c6-4c97-b2c2-8a36695848f2 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:18.070619 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-22aed6a6-71c6-4c97-b2c2-8a36695848f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:18.210557 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22aed6a6-71c6-4c97-b2c2-8a36695848f2 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:18.214918 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2cc4e2-3fa4-4a9e-905e-2506be7956f6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59be342541264942ad87f60ff4de6c8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:18.215550 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2cc4e2-3fa4-4a9e-905e-2506be7956f6 service cinder] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:18.215831 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2cc4e2-3fa4-4a9e-905e-2506be7956f6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:18.216104 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2cc4e2-3fa4-4a9e-905e-2506be7956f6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:18.220320 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-0e2cc4e2-3fa4-4a9e-905e-2506be7956f6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:18.267273 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2bff43-f50f-4698-b19a-8a22db5a042f None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:18.269450 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-4a2bff43-f50f-4698-b19a-8a22db5a042f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:18.367572 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2cc4e2-3fa4-4a9e-905e-2506be7956f6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:18.370139 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2cc4e2-3fa4-4a9e-905e-2506be7956f6 service cinder] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:18.393771 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 837/1671] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:40:18 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:40:18.441834 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2bff43-f50f-4698-b19a-8a22db5a042f None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:18.454267 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc7cf90-7a90-4daf-94b1-a8d2dd54a690 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:18.455589 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc7cf90-7a90-4daf-94b1-a8d2dd54a690 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:18.456247 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc7cf90-7a90-4daf-94b1-a8d2dd54a690 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:18.457143 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc7cf90-7a90-4daf-94b1-a8d2dd54a690 admin admin] PATH_INFO: `/v3/projects/24c5a88ec45a4a33b5fc19d0f4ca45bb` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:18.459382 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc7cf90-7a90-4daf-94b1-a8d2dd54a690 admin admin] RBAC: Authorizing `identity:delete_project(project_id=24c5a88ec45a4a33b5fc19d0f4ca45bb)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:18.462118 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc7cf90-7a90-4daf-94b1-a8d2dd54a690 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:18.564751 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-8cc7cf90-7a90-4daf-94b1-a8d2dd54a690 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '24c5a88ec45a4a33b5fc19d0f4ca45bb', 'request_id': 'req-8cc7cf90-7a90-4daf-94b1-a8d2dd54a690'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:18.582408 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-8cc7cf90-7a90-4daf-94b1-a8d2dd54a690 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '24c5a88ec45a4a33b5fc19d0f4ca45bb', 'request_id': 'req-8cc7cf90-7a90-4daf-94b1-a8d2dd54a690'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:18.584396 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:40:18.596999 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-8cc7cf90-7a90-4daf-94b1-a8d2dd54a690 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '24c5a88ec45a4a33b5fc19d0f4ca45bb', 'request_id': 'req-8cc7cf90-7a90-4daf-94b1-a8d2dd54a690'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:18.619545 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-8cc7cf90-7a90-4daf-94b1-a8d2dd54a690 admin admin] The token cache is being invalidate because project 24c5a88ec45a4a33b5fc19d0f4ca45bb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81758) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:40:18.620174 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-8cc7cf90-7a90-4daf-94b1-a8d2dd54a690 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:18.622979 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-8cc7cf90-7a90-4daf-94b1-a8d2dd54a690 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:18.626221 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 835/1672] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 17:40:18 2022] DELETE /identity/v3/projects/24c5a88ec45a4a33b5fc19d0f4ca45bb => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:40:18.631644 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882ad02e-6f79-4ba9-9c26-1302c7ed7fd5 None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:18.632156 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882ad02e-6f79-4ba9-9c26-1302c7ed7fd5 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:18.632741 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882ad02e-6f79-4ba9-9c26-1302c7ed7fd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:18.668797 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-882ad02e-6f79-4ba9-9c26-1302c7ed7fd5 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:40:18.673468 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-882ad02e-6f79-4ba9-9c26-1302c7ed7fd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:18.821607 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:40:18.834618 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 838/1673] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:40:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:40:18.843292 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a2fd73-c342-42ae-9164-b31d3eddbb7a None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:18.843808 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a2fd73-c342-42ae-9164-b31d3eddbb7a None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:18.844063 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a2fd73-c342-42ae-9164-b31d3eddbb7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:18.878090 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-75a2fd73-c342-42ae-9164-b31d3eddbb7a None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:40:18.880115 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-75a2fd73-c342-42ae-9164-b31d3eddbb7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:18.894957 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 836/1674] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:40:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:40:18.900813 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882ad02e-6f79-4ba9-9c26-1302c7ed7fd5 None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:18.926576 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882ad02e-6f79-4ba9-9c26-1302c7ed7fd5 None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:18.933119 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bae0b9a-9384-4adf-b7ff-c48a0abee1f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:18.934228 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bae0b9a-9384-4adf-b7ff-c48a0abee1f6 admin admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:18.934740 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bae0b9a-9384-4adf-b7ff-c48a0abee1f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:18.935232 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bae0b9a-9384-4adf-b7ff-c48a0abee1f6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:18.936101 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bae0b9a-9384-4adf-b7ff-c48a0abee1f6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:40:18.936711 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bae0b9a-9384-4adf-b7ff-c48a0abee1f6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:18.953762 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bae0b9a-9384-4adf-b7ff-c48a0abee1f6 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:18.991740 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 839/1675] 10.222.0.45 () {58 vars in 1153 bytes} [Wed Aug 24 17:40:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:40:18.995966 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a2fd73-c342-42ae-9164-b31d3eddbb7a None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:19.009311 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a2fd73-c342-42ae-9164-b31d3eddbb7a None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:19.021926 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e5c1bd-d487-4a8d-b98c-6453620d77eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:19.022713 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e5c1bd-d487-4a8d-b98c-6453620d77eb admin admin] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:19.023210 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e5c1bd-d487-4a8d-b98c-6453620d77eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:19.023564 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e5c1bd-d487-4a8d-b98c-6453620d77eb admin admin] PATH_INFO: `/v3/projects` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:19.024355 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e5c1bd-d487-4a8d-b98c-6453620d77eb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:19.028881 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e5c1bd-d487-4a8d-b98c-6453620d77eb admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:19.125088 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 837/1676] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 17:40:18 2022] POST /identity/v3/projects => generated 355 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:40:19.129891 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bae0b9a-9384-4adf-b7ff-c48a0abee1f6 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:19.151199 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bae0b9a-9384-4adf-b7ff-c48a0abee1f6 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:19.337438 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abf3899-8f1c-4e59-98a2-c702b1ae359e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:19.338080 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abf3899-8f1c-4e59-98a2-c702b1ae359e admin admin] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:19.338361 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abf3899-8f1c-4e59-98a2-c702b1ae359e admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:19.338723 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abf3899-8f1c-4e59-98a2-c702b1ae359e admin admin] PATH_INFO: `/v3/users` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:19.339277 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abf3899-8f1c-4e59-98a2-c702b1ae359e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:19.349294 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abf3899-8f1c-4e59-98a2-c702b1ae359e admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:19.404973 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 840/1677] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 17:40:19 2022] POST /identity/v3/users => generated 278 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:40:19.407172 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e5c1bd-d487-4a8d-b98c-6453620d77eb admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:19.423164 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e5c1bd-d487-4a8d-b98c-6453620d77eb admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:19.436079 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac49ea38-816c-4064-ad97-8caa1a25bdae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:19.439670 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac49ea38-816c-4064-ad97-8caa1a25bdae admin admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:19.439961 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac49ea38-816c-4064-ad97-8caa1a25bdae admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:19.440242 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac49ea38-816c-4064-ad97-8caa1a25bdae admin admin] PATH_INFO: `/v3/roles` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:19.440951 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac49ea38-816c-4064-ad97-8caa1a25bdae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:19.442585 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac49ea38-816c-4064-ad97-8caa1a25bdae admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:19.475744 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abf3899-8f1c-4e59-98a2-c702b1ae359e admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:19.482465 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 838/1678] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:40:19 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:40:19.487299 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abf3899-8f1c-4e59-98a2-c702b1ae359e admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:19.500340 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cc674b-be2f-4842-9aad-42121988ee95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:19.500750 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cc674b-be2f-4842-9aad-42121988ee95 admin admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:19.501041 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cc674b-be2f-4842-9aad-42121988ee95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:19.501848 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cc674b-be2f-4842-9aad-42121988ee95 admin admin] PATH_INFO: `/v3/roles` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:19.502646 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cc674b-be2f-4842-9aad-42121988ee95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:19.504162 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cc674b-be2f-4842-9aad-42121988ee95 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:19.547371 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 841/1679] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:40:19 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 17:40:19.551807 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac49ea38-816c-4064-ad97-8caa1a25bdae admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:19.571192 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac49ea38-816c-4064-ad97-8caa1a25bdae admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:19.585451 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71764f3c-efe1-40fb-b1dd-6714e79eb8c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:19.586546 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71764f3c-efe1-40fb-b1dd-6714e79eb8c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:19.586980 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71764f3c-efe1-40fb-b1dd-6714e79eb8c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:19.587415 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71764f3c-efe1-40fb-b1dd-6714e79eb8c9 admin admin] PATH_INFO: `/v3/projects/fb57982c79744bc8836897fb056d74da/users/2aefcf1d24db46d2a17f3953bc840094/roles/02476a5b82eb4b188c18a5d706a3ef68` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:19.621006 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71764f3c-efe1-40fb-b1dd-6714e79eb8c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb57982c79744bc8836897fb056d74da, user_id=2aefcf1d24db46d2a17f3953bc840094, role_id=02476a5b82eb4b188c18a5d706a3ef68)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:19.624099 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71764f3c-efe1-40fb-b1dd-6714e79eb8c9 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:19.653448 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 839/1680] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 17:40:19 2022] PUT /identity/v3/projects/fb57982c79744bc8836897fb056d74da/users/2aefcf1d24db46d2a17f3953bc840094/roles/02476a5b82eb4b188c18a5d706a3ef68 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:40:19.663585 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05990686-c7ca-433d-bd74-9a8d57a71ae8 None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:19.664492 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05990686-c7ca-433d-bd74-9a8d57a71ae8 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:19.664492 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05990686-c7ca-433d-bd74-9a8d57a71ae8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:19.705049 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-05990686-c7ca-433d-bd74-9a8d57a71ae8 None None] MFA Rules not processed for user `2aefcf1d24db46d2a17f3953bc840094`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:40:19.706153 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-05990686-c7ca-433d-bd74-9a8d57a71ae8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:19.889233 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 842/1681] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:40:19 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:40:19.894645 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71764f3c-efe1-40fb-b1dd-6714e79eb8c9 admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:20.022861 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71764f3c-efe1-40fb-b1dd-6714e79eb8c9 admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:20.040496 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fe56e0-8331-4a8d-a43c-5ad8e087c3cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:20.041169 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fe56e0-8331-4a8d-a43c-5ad8e087c3cb admin admin] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:20.041453 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fe56e0-8331-4a8d-a43c-5ad8e087c3cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:20.041733 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fe56e0-8331-4a8d-a43c-5ad8e087c3cb admin admin] PATH_INFO: `/v3/projects` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:20.042671 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fe56e0-8331-4a8d-a43c-5ad8e087c3cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:20.055001 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fe56e0-8331-4a8d-a43c-5ad8e087c3cb admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:20.116856 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 840/1682] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 17:40:19 2022] POST /identity/v3/projects => generated 355 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:40:20.117398 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05990686-c7ca-433d-bd74-9a8d57a71ae8 None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:20.134138 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05990686-c7ca-433d-bd74-9a8d57a71ae8 None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:20.219859 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:40:20.237422 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc2c7f5-f2e3-4cbc-95b9-18fe867281df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:20.238081 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc2c7f5-f2e3-4cbc-95b9-18fe867281df admin admin] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:20.238431 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc2c7f5-f2e3-4cbc-95b9-18fe867281df admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:20.238774 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc2c7f5-f2e3-4cbc-95b9-18fe867281df admin admin] PATH_INFO: `/v3/users` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:20.239956 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc2c7f5-f2e3-4cbc-95b9-18fe867281df admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:20.242928 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc2c7f5-f2e3-4cbc-95b9-18fe867281df admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:20.271655 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 843/1683] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 17:40:20 2022] POST /identity/v3/users => generated 284 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:40:20.275391 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fe56e0-8331-4a8d-a43c-5ad8e087c3cb admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:20.288887 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fe56e0-8331-4a8d-a43c-5ad8e087c3cb admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:20.304544 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba44faa-eb5d-4071-97d4-ac38caf8826a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:20.305204 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba44faa-eb5d-4071-97d4-ac38caf8826a admin admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:20.305495 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba44faa-eb5d-4071-97d4-ac38caf8826a admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:20.305775 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba44faa-eb5d-4071-97d4-ac38caf8826a admin admin] PATH_INFO: `/v3/roles` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:20.306601 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba44faa-eb5d-4071-97d4-ac38caf8826a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:20.310262 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba44faa-eb5d-4071-97d4-ac38caf8826a admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:20.335979 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 841/1684] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:40:20 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:40:20.340404 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc2c7f5-f2e3-4cbc-95b9-18fe867281df admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:20.351644 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc2c7f5-f2e3-4cbc-95b9-18fe867281df admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:20.356834 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f037bd-857c-4f3e-9da3-cd720c9e5ed1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:20.357608 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f037bd-857c-4f3e-9da3-cd720c9e5ed1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:20.358014 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f037bd-857c-4f3e-9da3-cd720c9e5ed1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:20.358850 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f037bd-857c-4f3e-9da3-cd720c9e5ed1 admin admin] PATH_INFO: `/v3/projects/afaae2a2b1d749deacbd8c2645bcb93f/users/019fa20317e74ef9ad1bb6ef28fd34fa/roles/e411ad9eb96e4e8d85b59b5b700ada7b` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:20.375860 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f037bd-857c-4f3e-9da3-cd720c9e5ed1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=afaae2a2b1d749deacbd8c2645bcb93f, user_id=019fa20317e74ef9ad1bb6ef28fd34fa, role_id=e411ad9eb96e4e8d85b59b5b700ada7b)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:20.378988 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f037bd-857c-4f3e-9da3-cd720c9e5ed1 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:20.397597 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 844/1685] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 17:40:20 2022] PUT /identity/v3/projects/afaae2a2b1d749deacbd8c2645bcb93f/users/019fa20317e74ef9ad1bb6ef28fd34fa/roles/e411ad9eb96e4e8d85b59b5b700ada7b => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:40:20.401334 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba44faa-eb5d-4071-97d4-ac38caf8826a admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:20.415030 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba44faa-eb5d-4071-97d4-ac38caf8826a admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:20.489810 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:40:20.509530 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74198545-44ff-4560-a2bd-c7c90fb425e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:20.511067 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74198545-44ff-4560-a2bd-c7c90fb425e0 admin admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:20.511476 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74198545-44ff-4560-a2bd-c7c90fb425e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:20.511857 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74198545-44ff-4560-a2bd-c7c90fb425e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:20.512812 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74198545-44ff-4560-a2bd-c7c90fb425e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:20.516880 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74198545-44ff-4560-a2bd-c7c90fb425e0 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:20.547870 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 842/1686] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:40:20 2022] GET /identity/v3/roles => generated 1420 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 17:40:20.549159 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f037bd-857c-4f3e-9da3-cd720c9e5ed1 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:20.561754 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f037bd-857c-4f3e-9da3-cd720c9e5ed1 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:20.569078 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db72828-6b85-42f8-b2e6-8109ba937676 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:20.569904 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db72828-6b85-42f8-b2e6-8109ba937676 admin admin] REQUEST_METHOD: `PUT` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:20.570650 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db72828-6b85-42f8-b2e6-8109ba937676 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:20.571020 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db72828-6b85-42f8-b2e6-8109ba937676 admin admin] PATH_INFO: `/v3/projects/afaae2a2b1d749deacbd8c2645bcb93f/users/019fa20317e74ef9ad1bb6ef28fd34fa/roles/e411ad9eb96e4e8d85b59b5b700ada7b` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:20.574513 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db72828-6b85-42f8-b2e6-8109ba937676 admin admin] RBAC: Authorizing `identity:create_grant(project_id=afaae2a2b1d749deacbd8c2645bcb93f, user_id=019fa20317e74ef9ad1bb6ef28fd34fa, role_id=e411ad9eb96e4e8d85b59b5b700ada7b)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:20.580895 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db72828-6b85-42f8-b2e6-8109ba937676 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:20.611435 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 845/1687] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 17:40:20 2022] PUT /identity/v3/projects/afaae2a2b1d749deacbd8c2645bcb93f/users/019fa20317e74ef9ad1bb6ef28fd34fa/roles/e411ad9eb96e4e8d85b59b5b700ada7b => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:40:20.613784 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9ff73c-df5d-4450-81e6-0e922b513959 None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:20.614591 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9ff73c-df5d-4450-81e6-0e922b513959 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:20.614591 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9ff73c-df5d-4450-81e6-0e922b513959 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:20.681127 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-6d9ff73c-df5d-4450-81e6-0e922b513959 None None] MFA Rules not processed for user `019fa20317e74ef9ad1bb6ef28fd34fa`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:40:20.686135 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-6d9ff73c-df5d-4450-81e6-0e922b513959 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:20.962267 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 843/1688] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:40:20 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 350 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:40:20.972845 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db72828-6b85-42f8-b2e6-8109ba937676 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:20.974962 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-4db72828-6b85-42f8-b2e6-8109ba937676 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:21.071273 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db72828-6b85-42f8-b2e6-8109ba937676 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:21.075977 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0016595-fdda-4422-81bc-dbd9b0f726a2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59be342541264942ad87f60ff4de6c8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:21.077088 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0016595-fdda-4422-81bc-dbd9b0f726a2 service cinder] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:21.077592 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0016595-fdda-4422-81bc-dbd9b0f726a2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:21.078408 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0016595-fdda-4422-81bc-dbd9b0f726a2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:21.090185 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0016595-fdda-4422-81bc-dbd9b0f726a2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:21.091749 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0016595-fdda-4422-81bc-dbd9b0f726a2 service cinder] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:21.114389 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 846/1689] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:40:20 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:40:21.220941 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9ff73c-df5d-4450-81e6-0e922b513959 None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:21.222967 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-6d9ff73c-df5d-4450-81e6-0e922b513959 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:21.239485 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9ff73c-df5d-4450-81e6-0e922b513959 None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:21.244042 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c55a119-3569-46d8-a3a6-54c3d7e67e8b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59be342541264942ad87f60ff4de6c8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:21.245045 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c55a119-3569-46d8-a3a6-54c3d7e67e8b service cinder] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:21.245160 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c55a119-3569-46d8-a3a6-54c3d7e67e8b service cinder] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:21.245407 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c55a119-3569-46d8-a3a6-54c3d7e67e8b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:21.377720 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c55a119-3569-46d8-a3a6-54c3d7e67e8b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:21.379186 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c55a119-3569-46d8-a3a6-54c3d7e67e8b service cinder] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:21.406318 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 844/1690] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:40:21 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:40:22.342775 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57396b78-23fc-49e9-a1ae-5650a8cefd45 None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:22.342775 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57396b78-23fc-49e9-a1ae-5650a8cefd45 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:22.342775 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57396b78-23fc-49e9-a1ae-5650a8cefd45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:22.437682 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-57396b78-23fc-49e9-a1ae-5650a8cefd45 None None] MFA Rules not processed for user `40c0367417434c04948e9f2a03e880a9`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:40:22.439571 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-57396b78-23fc-49e9-a1ae-5650a8cefd45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:22.628603 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 847/1691] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:40:22 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:40:22.636313 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71c76a3-d860-4348-83d7-4e89cd6258fb None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:22.637264 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71c76a3-d860-4348-83d7-4e89cd6258fb None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:22.637750 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71c76a3-d860-4348-83d7-4e89cd6258fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:22.680103 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-b71c76a3-d860-4348-83d7-4e89cd6258fb None None] MFA Rules not processed for user `40c0367417434c04948e9f2a03e880a9`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:40:22.692828 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-b71c76a3-d860-4348-83d7-4e89cd6258fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:22.714881 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 845/1692] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:40:22 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:40:22.726520 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57396b78-23fc-49e9-a1ae-5650a8cefd45 None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:22.728128 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-57396b78-23fc-49e9-a1ae-5650a8cefd45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:22.817126 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:40:22.855616 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57396b78-23fc-49e9-a1ae-5650a8cefd45 None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:22.861711 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0648674f-30c1-4dd4-9a70-6cc22aaad6a4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '053b7ce6d6f649ed91a26e434dab805e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:22.862816 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0648674f-30c1-4dd4-9a70-6cc22aaad6a4 service nova] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:22.862816 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0648674f-30c1-4dd4-9a70-6cc22aaad6a4 service nova] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:22.863126 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0648674f-30c1-4dd4-9a70-6cc22aaad6a4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:22.874951 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0648674f-30c1-4dd4-9a70-6cc22aaad6a4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:22.877298 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0648674f-30c1-4dd4-9a70-6cc22aaad6a4 service nova] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:22.928981 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 848/1693] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:40:22 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:40:23.847942 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcffad4-46af-41ef-bfb5-6a548c7c11d3 None None] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:23.848469 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcffad4-46af-41ef-bfb5-6a548c7c11d3 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:23.848469 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcffad4-46af-41ef-bfb5-6a548c7c11d3 None None] PATH_INFO: `/` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:23.850177 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 846/1694] 10.222.0.45 () {54 vars in 912 bytes} [Wed Aug 24 17:40:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:40:23.856702 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143b684a-1900-4001-bfff-7299839b2939 None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:23.857156 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143b684a-1900-4001-bfff-7299839b2939 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:23.857406 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143b684a-1900-4001-bfff-7299839b2939 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:23.972842 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-143b684a-1900-4001-bfff-7299839b2939 None None] MFA Rules not processed for user `43ad98a250a144099ccd18378aedd4b5`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:40:23.974228 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-143b684a-1900-4001-bfff-7299839b2939 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:24.068140 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 849/1695] 10.222.0.45 () {58 vars in 1009 bytes} [Wed Aug 24 17:40:23 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:40:24.074531 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcffad4-46af-41ef-bfb5-6a548c7c11d3 None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:24.076349 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-7bcffad4-46af-41ef-bfb5-6a548c7c11d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:24.112708 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcffad4-46af-41ef-bfb5-6a548c7c11d3 None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:24.119387 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83489d19-d842-475a-bfb5-4b339b61356f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43ad98a250a144099ccd18378aedd4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:24.120230 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83489d19-d842-475a-bfb5-4b339b61356f service neutron] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:24.120687 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83489d19-d842-475a-bfb5-4b339b61356f service neutron] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:24.122343 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83489d19-d842-475a-bfb5-4b339b61356f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:24.137129 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83489d19-d842-475a-bfb5-4b339b61356f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:24.138987 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83489d19-d842-475a-bfb5-4b339b61356f service neutron] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:24.165842 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 847/1696] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:40:24 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:40:25.204993 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0295aac9-9a50-4a18-baea-02128b978d8c None None] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:25.205821 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0295aac9-9a50-4a18-baea-02128b978d8c None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:25.206271 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0295aac9-9a50-4a18-baea-02128b978d8c None None] PATH_INFO: `/` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:25.208616 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 850/1697] 10.222.0.45 () {54 vars in 961 bytes} [Wed Aug 24 17:40:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:40:25.215206 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d8bd98-f946-42d6-9e54-1c783efe63a6 None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:25.216317 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d8bd98-f946-42d6-9e54-1c783efe63a6 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:25.216828 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d8bd98-f946-42d6-9e54-1c783efe63a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:25.273740 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-b5d8bd98-f946-42d6-9e54-1c783efe63a6 None None] MFA Rules not processed for user `37def4721b554b9ebb7704a19642d15b`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:40:25.276512 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-b5d8bd98-f946-42d6-9e54-1c783efe63a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:25.355240 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9a3b61-3c80-4e6c-ba8e-35c9b3dfb6c9 None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:25.355725 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9a3b61-3c80-4e6c-ba8e-35c9b3dfb6c9 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:25.356900 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9a3b61-3c80-4e6c-ba8e-35c9b3dfb6c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:25.422879 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:40:25.451864 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-4c9a3b61-3c80-4e6c-ba8e-35c9b3dfb6c9 None None] MFA Rules not processed for user `40c0367417434c04948e9f2a03e880a9`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:40:25.463589 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-4c9a3b61-3c80-4e6c-ba8e-35c9b3dfb6c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:25.479398 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 848/1698] 10.222.0.45 () {58 vars in 1058 bytes} [Wed Aug 24 17:40:25 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:40:25.498856 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 851/1699] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:40:25 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:40:25.521072 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d8bd98-f946-42d6-9e54-1c783efe63a6 None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:25.535989 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d8bd98-f946-42d6-9e54-1c783efe63a6 None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:25.561339 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befc0fd3-a524-465a-a1c0-097d28132633 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37def4721b554b9ebb7704a19642d15b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:25.562197 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befc0fd3-a524-465a-a1c0-097d28132633 service placement] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:25.562640 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befc0fd3-a524-465a-a1c0-097d28132633 service placement] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:25.563057 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befc0fd3-a524-465a-a1c0-097d28132633 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:25.565124 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-befc0fd3-a524-465a-a1c0-097d28132633 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:25.604175 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befc0fd3-a524-465a-a1c0-097d28132633 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:25.606427 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befc0fd3-a524-465a-a1c0-097d28132633 service placement] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:25.641546 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 849/1700] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:40:25 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:40:26.593580 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839cbfde-d6fd-45d6-a8a2-9a3575a4dbed None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:26.594187 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839cbfde-d6fd-45d6-a8a2-9a3575a4dbed None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:26.594187 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839cbfde-d6fd-45d6-a8a2-9a3575a4dbed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:26.594666 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c665520-cab3-4039-944f-493666ecabce None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:26.595738 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c665520-cab3-4039-944f-493666ecabce None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:26.596207 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c665520-cab3-4039-944f-493666ecabce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:26.679010 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-839cbfde-d6fd-45d6-a8a2-9a3575a4dbed None None] MFA Rules not processed for user `c36a9a0d92024734b1aa007a56c6328c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:40:26.682874 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-839cbfde-d6fd-45d6-a8a2-9a3575a4dbed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:26.715027 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-5c665520-cab3-4039-944f-493666ecabce None None] MFA Rules not processed for user `1d0ae308269b4cdd80990f7b38c0cd0c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:40:26.723549 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-5c665520-cab3-4039-944f-493666ecabce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:26.977910 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 850/1701] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:40:26 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 385 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:40:26.979838 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d079ba48-8014-4977-aa53-b06bd200c383 None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:26.982696 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d079ba48-8014-4977-aa53-b06bd200c383 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:26.982696 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d079ba48-8014-4977-aa53-b06bd200c383 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:27.018893 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 852/1702] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:40:26 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 427 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:40:27.034832 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839cbfde-d6fd-45d6-a8a2-9a3575a4dbed None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:27.036891 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-d079ba48-8014-4977-aa53-b06bd200c383 None None] MFA Rules not processed for user `1d0ae308269b4cdd80990f7b38c0cd0c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:40:27.038151 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-d079ba48-8014-4977-aa53-b06bd200c383 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:27.054171 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 851/1703] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:40:26 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:40:27.059966 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839cbfde-d6fd-45d6-a8a2-9a3575a4dbed None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:27.067522 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d079ba48-8014-4977-aa53-b06bd200c383 None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:27.069887 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-d079ba48-8014-4977-aa53-b06bd200c383 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:27.074836 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e594445-7e1e-4848-a2cc-ae707416f870 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '053b7ce6d6f649ed91a26e434dab805e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:27.075585 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e594445-7e1e-4848-a2cc-ae707416f870 service nova] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:27.075868 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e594445-7e1e-4848-a2cc-ae707416f870 service nova] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:27.076211 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e594445-7e1e-4848-a2cc-ae707416f870 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:27.114936 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e594445-7e1e-4848-a2cc-ae707416f870 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:27.116947 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e594445-7e1e-4848-a2cc-ae707416f870 service nova] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:27.124252 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d079ba48-8014-4977-aa53-b06bd200c383 None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:27.141071 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 853/1704] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:40:27 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:40:27.149916 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0450f26-a75b-4159-90a4-995991ca4ca8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '053b7ce6d6f649ed91a26e434dab805e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:27.150802 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0450f26-a75b-4159-90a4-995991ca4ca8 service nova] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:27.151969 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0450f26-a75b-4159-90a4-995991ca4ca8 service nova] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:27.152255 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0450f26-a75b-4159-90a4-995991ca4ca8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:27.189238 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0450f26-a75b-4159-90a4-995991ca4ca8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:27.190868 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0450f26-a75b-4159-90a4-995991ca4ca8 service nova] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:27.198466 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e594445-7e1e-4848-a2cc-ae707416f870 service nova] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:27.218613 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e594445-7e1e-4848-a2cc-ae707416f870 service nova] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:27.227947 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebcca57-eb16-407d-aada-43828662e261 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59be342541264942ad87f60ff4de6c8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:27.229463 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebcca57-eb16-407d-aada-43828662e261 service cinder] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:27.229964 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebcca57-eb16-407d-aada-43828662e261 service cinder] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:27.230457 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebcca57-eb16-407d-aada-43828662e261 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:27.239165 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-8ebcca57-eb16-407d-aada-43828662e261 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:27.246601 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 852/1705] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:40:27 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:40:27.325564 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:40:27.410575 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebcca57-eb16-407d-aada-43828662e261 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:27.415073 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebcca57-eb16-407d-aada-43828662e261 service cinder] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:27.485597 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 854/1706] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:40:27 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:40:27.603012 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1f23b2-66d0-4914-9942-7098a290e553 None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:27.603854 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1f23b2-66d0-4914-9942-7098a290e553 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:27.604264 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1f23b2-66d0-4914-9942-7098a290e553 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:27.695072 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-ab1f23b2-66d0-4914-9942-7098a290e553 None None] MFA Rules not processed for user `892dad01ea684e6cb2d5cec659c6c4b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:40:27.698462 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-ab1f23b2-66d0-4914-9942-7098a290e553 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:27.867783 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebcca57-eb16-407d-aada-43828662e261 service cinder] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:27.870793 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-8ebcca57-eb16-407d-aada-43828662e261 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:27.969782 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 853/1707] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:40:27 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 369 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:40:28.075873 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebcca57-eb16-407d-aada-43828662e261 service cinder] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:28.101879 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae2968c-d0b9-4257-8e68-3f2719168fdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:28.102791 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae2968c-d0b9-4257-8e68-3f2719168fdd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:28.104581 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae2968c-d0b9-4257-8e68-3f2719168fdd admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:28.105011 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae2968c-d0b9-4257-8e68-3f2719168fdd admin admin] PATH_INFO: `/v3/users/2d27ff5caebd413caf117b61e1f74c8a` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:28.109252 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae2968c-d0b9-4257-8e68-3f2719168fdd admin admin] RBAC: Authorizing `identity:delete_user(user_id=2d27ff5caebd413caf117b61e1f74c8a)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:28.116509 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae2968c-d0b9-4257-8e68-3f2719168fdd admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:28.258891 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-9ae2968c-d0b9-4257-8e68-3f2719168fdd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2d27ff5caebd413caf117b61e1f74c8a'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:28.261023 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-9ae2968c-d0b9-4257-8e68-3f2719168fdd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2d27ff5caebd413caf117b61e1f74c8a'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:28.285808 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-9ae2968c-d0b9-4257-8e68-3f2719168fdd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2d27ff5caebd413caf117b61e1f74c8a'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:28.289247 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-9ae2968c-d0b9-4257-8e68-3f2719168fdd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2d27ff5caebd413caf117b61e1f74c8a'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:28.335388 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-9ae2968c-d0b9-4257-8e68-3f2719168fdd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2d27ff5caebd413caf117b61e1f74c8a'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:28.397411 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 855/1708] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 17:40:27 2022] DELETE /identity/v3/users/2d27ff5caebd413caf117b61e1f74c8a => generated 0 bytes in 527 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:40:28.399413 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaf048e-4681-40dd-9cfe-4903e19c5ee8 None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:28.399835 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaf048e-4681-40dd-9cfe-4903e19c5ee8 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:28.400833 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaf048e-4681-40dd-9cfe-4903e19c5ee8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:28.457967 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-3eaf048e-4681-40dd-9cfe-4903e19c5ee8 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:40:28.459120 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-3eaf048e-4681-40dd-9cfe-4903e19c5ee8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:28.556012 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae2968c-d0b9-4257-8e68-3f2719168fdd admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:28.558110 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-9ae2968c-d0b9-4257-8e68-3f2719168fdd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:28.573750 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:40:28.672142 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 854/1709] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:40:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:40:28.715346 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eaf048e-4681-40dd-9cfe-4903e19c5ee8 None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:28.717713 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-3eaf048e-4681-40dd-9cfe-4903e19c5ee8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:28.805114 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:40:28.927420 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae2968c-d0b9-4257-8e68-3f2719168fdd admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:28.953705 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eaf048e-4681-40dd-9cfe-4903e19c5ee8 None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:28.954009 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb00543-814f-4111-ad92-4aaeb79e8a70 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '053b7ce6d6f649ed91a26e434dab805e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:28.954896 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb00543-814f-4111-ad92-4aaeb79e8a70 service nova] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:28.955110 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb00543-814f-4111-ad92-4aaeb79e8a70 service nova] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:28.955544 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb00543-814f-4111-ad92-4aaeb79e8a70 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:28.961308 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c02cff8-6234-4f55-8178-8e954c7c11ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43ad98a250a144099ccd18378aedd4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:28.961940 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c02cff8-6234-4f55-8178-8e954c7c11ce service neutron] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:28.962279 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c02cff8-6234-4f55-8178-8e954c7c11ce service neutron] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:28.962524 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c02cff8-6234-4f55-8178-8e954c7c11ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:28.964970 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-4fb00543-814f-4111-ad92-4aaeb79e8a70 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:28.993839 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c02cff8-6234-4f55-8178-8e954c7c11ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:28.995684 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c02cff8-6234-4f55-8178-8e954c7c11ce service neutron] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:29.039771 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 855/1710] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:40:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 17:40:29.042515 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ec9edf-5272-4f90-8654-1eb84ec80cf1 None None] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:29.042808 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ec9edf-5272-4f90-8654-1eb84ec80cf1 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:29.043086 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ec9edf-5272-4f90-8654-1eb84ec80cf1 None None] PATH_INFO: `/` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:29.047477 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 856/1711] 10.222.0.45 () {54 vars in 912 bytes} [Wed Aug 24 17:40:29 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:40:29.067079 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304ea916-992c-4cbe-b3cf-88801c5415c2 None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:29.067375 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304ea916-992c-4cbe-b3cf-88801c5415c2 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:29.078070 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304ea916-992c-4cbe-b3cf-88801c5415c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:29.144152 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb00543-814f-4111-ad92-4aaeb79e8a70 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:29.146249 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb00543-814f-4111-ad92-4aaeb79e8a70 service nova] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:29.166407 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 856/1712] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:40:28 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 611 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:40:29.169544 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-304ea916-992c-4cbe-b3cf-88801c5415c2 None None] MFA Rules not processed for user `43ad98a250a144099ccd18378aedd4b5`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:40:29.177132 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-304ea916-992c-4cbe-b3cf-88801c5415c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:29.195116 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 857/1713] 10.222.0.45 () {58 vars in 1009 bytes} [Wed Aug 24 17:40:29 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:40:29.247501 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb00543-814f-4111-ad92-4aaeb79e8a70 service nova] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:29.249148 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-4fb00543-814f-4111-ad92-4aaeb79e8a70 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:29.270235 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb00543-814f-4111-ad92-4aaeb79e8a70 service nova] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:29.288143 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aaa3c0d-1ba8-47ff-b19b-9f7d2317965b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43ad98a250a144099ccd18378aedd4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:29.288812 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaa3c0d-1ba8-47ff-b19b-9f7d2317965b service neutron] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:29.289376 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaa3c0d-1ba8-47ff-b19b-9f7d2317965b service neutron] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:29.289680 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaa3c0d-1ba8-47ff-b19b-9f7d2317965b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:29.304897 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aaa3c0d-1ba8-47ff-b19b-9f7d2317965b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:29.312243 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aaa3c0d-1ba8-47ff-b19b-9f7d2317965b service neutron] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:29.344727 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 857/1714] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:40:29 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:40:30.300393 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304ea916-992c-4cbe-b3cf-88801c5415c2 None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:30.301874 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-304ea916-992c-4cbe-b3cf-88801c5415c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:30.336839 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304ea916-992c-4cbe-b3cf-88801c5415c2 None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:30.343685 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edf8876-0a05-4e0e-af15-f2d350deba45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:30.344570 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edf8876-0a05-4e0e-af15-f2d350deba45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:30.344994 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edf8876-0a05-4e0e-af15-f2d350deba45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:30.345442 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edf8876-0a05-4e0e-af15-f2d350deba45 admin admin] PATH_INFO: `/v3/projects/3cbb522709a64ec1abdf99ede28bcae6` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:30.347455 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edf8876-0a05-4e0e-af15-f2d350deba45 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3cbb522709a64ec1abdf99ede28bcae6)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:30.349797 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edf8876-0a05-4e0e-af15-f2d350deba45 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:30.467474 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-6edf8876-0a05-4e0e-af15-f2d350deba45 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3cbb522709a64ec1abdf99ede28bcae6', 'request_id': 'req-6edf8876-0a05-4e0e-af15-f2d350deba45'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:30.496805 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-6edf8876-0a05-4e0e-af15-f2d350deba45 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3cbb522709a64ec1abdf99ede28bcae6', 'request_id': 'req-6edf8876-0a05-4e0e-af15-f2d350deba45'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:30.513873 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-6edf8876-0a05-4e0e-af15-f2d350deba45 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3cbb522709a64ec1abdf99ede28bcae6', 'request_id': 'req-6edf8876-0a05-4e0e-af15-f2d350deba45'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:30.553746 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-6edf8876-0a05-4e0e-af15-f2d350deba45 admin admin] The token cache is being invalidate because project 3cbb522709a64ec1abdf99ede28bcae6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81758) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:40:30.554371 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-6edf8876-0a05-4e0e-af15-f2d350deba45 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:30.555181 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-6edf8876-0a05-4e0e-af15-f2d350deba45 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:30.563051 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bb8b7f-ac7b-45af-bd22-436a02125530 None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:30.563510 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bb8b7f-ac7b-45af-bd22-436a02125530 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:30.564162 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bb8b7f-ac7b-45af-bd22-436a02125530 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:30.564796 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 858/1715] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 17:40:30 2022] DELETE /identity/v3/projects/3cbb522709a64ec1abdf99ede28bcae6 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:40:30.601806 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-56bb8b7f-ac7b-45af-bd22-436a02125530 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:40:30.606124 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-56bb8b7f-ac7b-45af-bd22-436a02125530 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:30.786119 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 858/1716] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:40:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:40:30.793776 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d70c56f-8f29-4a78-b4a2-56fd8e22a6b9 None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:30.794408 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d70c56f-8f29-4a78-b4a2-56fd8e22a6b9 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:30.795044 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d70c56f-8f29-4a78-b4a2-56fd8e22a6b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:30.837997 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-2d70c56f-8f29-4a78-b4a2-56fd8e22a6b9 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:40:30.863355 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-2d70c56f-8f29-4a78-b4a2-56fd8e22a6b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:30.901775 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bb8b7f-ac7b-45af-bd22-436a02125530 None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:30.907358 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 859/1717] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:40:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:40:30.926396 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bb8b7f-ac7b-45af-bd22-436a02125530 None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:30.940717 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3bc70a-6d5c-406a-a763-51e99888b205 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:30.941324 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3bc70a-6d5c-406a-a763-51e99888b205 admin admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:30.941775 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3bc70a-6d5c-406a-a763-51e99888b205 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:30.942192 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3bc70a-6d5c-406a-a763-51e99888b205 admin admin] PATH_INFO: `/v3/domains` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:30.943149 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3bc70a-6d5c-406a-a763-51e99888b205 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:40:30.943597 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3bc70a-6d5c-406a-a763-51e99888b205 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:30.945712 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3bc70a-6d5c-406a-a763-51e99888b205 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:30.998512 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d70c56f-8f29-4a78-b4a2-56fd8e22a6b9 None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:31.021631 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 859/1718] 10.222.0.45 () {58 vars in 1153 bytes} [Wed Aug 24 17:40:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:40:31.024229 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d70c56f-8f29-4a78-b4a2-56fd8e22a6b9 None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:31.072890 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74547b80-ca79-4643-89e7-d90352a9775b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:31.073524 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74547b80-ca79-4643-89e7-d90352a9775b admin admin] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:31.073808 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74547b80-ca79-4643-89e7-d90352a9775b admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:31.074082 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74547b80-ca79-4643-89e7-d90352a9775b admin admin] PATH_INFO: `/v3/projects` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:31.074723 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74547b80-ca79-4643-89e7-d90352a9775b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:31.094280 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74547b80-ca79-4643-89e7-d90352a9775b admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:31.216140 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 860/1719] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 17:40:30 2022] POST /identity/v3/projects => generated 375 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:40:31.217674 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3bc70a-6d5c-406a-a763-51e99888b205 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:31.240114 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3bc70a-6d5c-406a-a763-51e99888b205 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:31.444108 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:40:31.473248 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422f481d-4bbf-43dd-a2fd-bbbbfc710309 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:31.474098 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422f481d-4bbf-43dd-a2fd-bbbbfc710309 admin admin] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:31.474670 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422f481d-4bbf-43dd-a2fd-bbbbfc710309 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:31.475119 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422f481d-4bbf-43dd-a2fd-bbbbfc710309 admin admin] PATH_INFO: `/v3/users` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:31.475918 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422f481d-4bbf-43dd-a2fd-bbbbfc710309 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:31.479158 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422f481d-4bbf-43dd-a2fd-bbbbfc710309 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:31.544031 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 860/1720] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 17:40:31 2022] POST /identity/v3/users => generated 288 bytes in 327 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:40:31.548041 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74547b80-ca79-4643-89e7-d90352a9775b admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:31.563734 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74547b80-ca79-4643-89e7-d90352a9775b admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:31.583086 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56433f07-42ce-4a2a-bd32-9a73cbce0715 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:31.583964 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56433f07-42ce-4a2a-bd32-9a73cbce0715 admin admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:31.584313 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56433f07-42ce-4a2a-bd32-9a73cbce0715 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:31.584689 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56433f07-42ce-4a2a-bd32-9a73cbce0715 admin admin] PATH_INFO: `/v3/roles` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:31.585410 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56433f07-42ce-4a2a-bd32-9a73cbce0715 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:31.587438 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56433f07-42ce-4a2a-bd32-9a73cbce0715 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:31.617536 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 861/1721] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:40:31 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:40:31.621282 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422f481d-4bbf-43dd-a2fd-bbbbfc710309 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:31.636924 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422f481d-4bbf-43dd-a2fd-bbbbfc710309 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:31.665255 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77eca825-409f-4fb9-8941-a45fc6b5b7d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:31.666468 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77eca825-409f-4fb9-8941-a45fc6b5b7d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:31.667486 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77eca825-409f-4fb9-8941-a45fc6b5b7d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:31.669801 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77eca825-409f-4fb9-8941-a45fc6b5b7d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:31.670617 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77eca825-409f-4fb9-8941-a45fc6b5b7d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:31.675110 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77eca825-409f-4fb9-8941-a45fc6b5b7d7 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:31.703889 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 861/1722] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:40:31 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:40:31.706617 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56433f07-42ce-4a2a-bd32-9a73cbce0715 admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:31.723540 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56433f07-42ce-4a2a-bd32-9a73cbce0715 admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:31.756797 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eef9a82-96db-483d-9e71-cdaf8b430120 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:31.757782 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eef9a82-96db-483d-9e71-cdaf8b430120 admin admin] REQUEST_METHOD: `PUT` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:31.758208 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eef9a82-96db-483d-9e71-cdaf8b430120 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:31.758731 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eef9a82-96db-483d-9e71-cdaf8b430120 admin admin] PATH_INFO: `/v3/projects/a1316765024f429bae881e682da29c80/users/0b053021680047488ee4e3bc75681cc7/roles/02476a5b82eb4b188c18a5d706a3ef68` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:31.801110 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eef9a82-96db-483d-9e71-cdaf8b430120 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1316765024f429bae881e682da29c80, user_id=0b053021680047488ee4e3bc75681cc7, role_id=02476a5b82eb4b188c18a5d706a3ef68)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:31.804853 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eef9a82-96db-483d-9e71-cdaf8b430120 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:31.811351 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:40:31.832023 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 862/1723] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 17:40:31 2022] PUT /identity/v3/projects/a1316765024f429bae881e682da29c80/users/0b053021680047488ee4e3bc75681cc7/roles/02476a5b82eb4b188c18a5d706a3ef68 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:40:31.844021 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b4a2b8-4aea-4c79-bbd6-cc9eb36433b0 None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:31.844646 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b4a2b8-4aea-4c79-bbd6-cc9eb36433b0 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:31.845039 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b4a2b8-4aea-4c79-bbd6-cc9eb36433b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:31.920066 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-47b4a2b8-4aea-4c79-bbd6-cc9eb36433b0 None None] MFA Rules not processed for user `0b053021680047488ee4e3bc75681cc7`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:40:31.921370 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-47b4a2b8-4aea-4c79-bbd6-cc9eb36433b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:32.077930 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 862/1724] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:40:31 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:40:32.085393 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eef9a82-96db-483d-9e71-cdaf8b430120 admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:32.087629 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-9eef9a82-96db-483d-9e71-cdaf8b430120 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:32.198487 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3011514a-f6eb-4f7c-9964-92bc5171f33c None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:32.199141 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3011514a-f6eb-4f7c-9964-92bc5171f33c None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:32.199750 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3011514a-f6eb-4f7c-9964-92bc5171f33c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:32.231428 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eef9a82-96db-483d-9e71-cdaf8b430120 admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:32.238907 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f76940a-8f52-4b52-88ce-56034aa8a231 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '053b7ce6d6f649ed91a26e434dab805e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:32.239817 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f76940a-8f52-4b52-88ce-56034aa8a231 service nova] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:32.240176 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f76940a-8f52-4b52-88ce-56034aa8a231 service nova] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:32.240528 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f76940a-8f52-4b52-88ce-56034aa8a231 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:32.252594 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f76940a-8f52-4b52-88ce-56034aa8a231 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:32.254625 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f76940a-8f52-4b52-88ce-56034aa8a231 service nova] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:32.257675 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-3011514a-f6eb-4f7c-9964-92bc5171f33c None None] MFA Rules not processed for user `c36a9a0d92024734b1aa007a56c6328c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:40:32.259602 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-3011514a-f6eb-4f7c-9964-92bc5171f33c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:32.266568 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 863/1725] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:40:32 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:40:32.445733 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 863/1726] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:40:32 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:40:32.601797 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f76940a-8f52-4b52-88ce-56034aa8a231 service nova] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:32.756792 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f76940a-8f52-4b52-88ce-56034aa8a231 service nova] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:32.763909 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d22edd1-f6d1-4f37-bbb8-fc38390e20aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:32.764953 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d22edd1-f6d1-4f37-bbb8-fc38390e20aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:32.765435 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d22edd1-f6d1-4f37-bbb8-fc38390e20aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:32.765904 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d22edd1-f6d1-4f37-bbb8-fc38390e20aa admin admin] PATH_INFO: `/v3/users/0b053021680047488ee4e3bc75681cc7` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:32.767864 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d22edd1-f6d1-4f37-bbb8-fc38390e20aa admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b053021680047488ee4e3bc75681cc7)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:32.779969 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d22edd1-f6d1-4f37-bbb8-fc38390e20aa admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:32.890238 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-8d22edd1-f6d1-4f37-bbb8-fc38390e20aa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b053021680047488ee4e3bc75681cc7'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:32.891108 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-8d22edd1-f6d1-4f37-bbb8-fc38390e20aa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b053021680047488ee4e3bc75681cc7'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:32.919514 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-8d22edd1-f6d1-4f37-bbb8-fc38390e20aa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b053021680047488ee4e3bc75681cc7'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:32.920634 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-8d22edd1-f6d1-4f37-bbb8-fc38390e20aa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b053021680047488ee4e3bc75681cc7'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:32.989518 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-8d22edd1-f6d1-4f37-bbb8-fc38390e20aa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b053021680047488ee4e3bc75681cc7'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:33.003626 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:INSERT=2,keystone:SELECT=88,keystone:DELETE=10 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:40:33.016472 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 864/1727] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 17:40:32 2022] DELETE /identity/v3/users/0b053021680047488ee4e3bc75681cc7 => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:40:33.022624 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608f8c06-3de0-41f2-b282-3c773f24173b None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:33.023659 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608f8c06-3de0-41f2-b282-3c773f24173b None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:33.025203 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608f8c06-3de0-41f2-b282-3c773f24173b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:33.048389 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d338d8-7977-40dc-90e6-82ba70cae072 None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:33.048873 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d338d8-7977-40dc-90e6-82ba70cae072 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:33.049291 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d338d8-7977-40dc-90e6-82ba70cae072 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:33.054606 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-608f8c06-3de0-41f2-b282-3c773f24173b None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:40:33.056059 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-608f8c06-3de0-41f2-b282-3c773f24173b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:33.130981 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-43d338d8-7977-40dc-90e6-82ba70cae072 None None] MFA Rules not processed for user `ea143d042ee94ae7a4544d90d18db7db`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:40:33.132876 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-43d338d8-7977-40dc-90e6-82ba70cae072 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:33.209149 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 864/1728] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:40:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:40:33.311064 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 865/1729] 10.222.0.45 () {58 vars in 1009 bytes} [Wed Aug 24 17:40:33 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:40:33.560295 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608f8c06-3de0-41f2-b282-3c773f24173b None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:33.561893 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-608f8c06-3de0-41f2-b282-3c773f24173b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:33.576333 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:40:33.641200 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d338d8-7977-40dc-90e6-82ba70cae072 None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:33.643789 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-43d338d8-7977-40dc-90e6-82ba70cae072 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:33.675190 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608f8c06-3de0-41f2-b282-3c773f24173b None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:33.681102 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d4f4ba-6671-46fd-a77d-7dbb06719375 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90f0772b64c84faaa689e7272fa97898', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:33.682365 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d4f4ba-6671-46fd-a77d-7dbb06719375 service swift] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:33.683791 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d4f4ba-6671-46fd-a77d-7dbb06719375 service swift] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:33.684156 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d4f4ba-6671-46fd-a77d-7dbb06719375 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:33.723352 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d4f4ba-6671-46fd-a77d-7dbb06719375 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:33.727671 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d4f4ba-6671-46fd-a77d-7dbb06719375 service swift] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:33.742852 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d338d8-7977-40dc-90e6-82ba70cae072 None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:33.755946 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa75193-cb41-4554-a5eb-b4d451eb83ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43ad98a250a144099ccd18378aedd4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:33.758474 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa75193-cb41-4554-a5eb-b4d451eb83ee service neutron] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:33.761549 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa75193-cb41-4554-a5eb-b4d451eb83ee service neutron] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:33.762194 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa75193-cb41-4554-a5eb-b4d451eb83ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:33.765646 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 865/1730] 10.222.0.45 () {60 vars in 1384 bytes} [Wed Aug 24 17:40:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 17:40:33.804327 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492df1ab-22b4-4a4d-a445-21eab6b4e3c8 None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:33.804327 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492df1ab-22b4-4a4d-a445-21eab6b4e3c8 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:33.808873 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa75193-cb41-4554-a5eb-b4d451eb83ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:33.811348 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492df1ab-22b4-4a4d-a445-21eab6b4e3c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:33.813556 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa75193-cb41-4554-a5eb-b4d451eb83ee service neutron] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:33.826118 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 866/1731] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:40:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 17:40:33.883168 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa75193-cb41-4554-a5eb-b4d451eb83ee service neutron] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:33.885182 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-8fa75193-cb41-4554-a5eb-b4d451eb83ee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:33.900856 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-492df1ab-22b4-4a4d-a445-21eab6b4e3c8 None None] MFA Rules not processed for user `892dad01ea684e6cb2d5cec659c6c4b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:40:33.902536 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-492df1ab-22b4-4a4d-a445-21eab6b4e3c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:33.932316 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa75193-cb41-4554-a5eb-b4d451eb83ee service neutron] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:33.973902 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678fe433-cbb3-480c-8b07-247c0522ceb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:33.982016 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678fe433-cbb3-480c-8b07-247c0522ceb9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:33.983297 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678fe433-cbb3-480c-8b07-247c0522ceb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:33.983664 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678fe433-cbb3-480c-8b07-247c0522ceb9 admin admin] PATH_INFO: `/v3/users/d2a240ceae7d424f8c2bda4b534eeb98` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:33.990026 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678fe433-cbb3-480c-8b07-247c0522ceb9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d2a240ceae7d424f8c2bda4b534eeb98)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:34.005000 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678fe433-cbb3-480c-8b07-247c0522ceb9 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:34.192004 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-678fe433-cbb3-480c-8b07-247c0522ceb9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd2a240ceae7d424f8c2bda4b534eeb98'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:34.199785 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-678fe433-cbb3-480c-8b07-247c0522ceb9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd2a240ceae7d424f8c2bda4b534eeb98'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:34.226242 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-678fe433-cbb3-480c-8b07-247c0522ceb9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd2a240ceae7d424f8c2bda4b534eeb98'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:34.231272 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-678fe433-cbb3-480c-8b07-247c0522ceb9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd2a240ceae7d424f8c2bda4b534eeb98'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:34.238583 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 866/1732] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:40:33 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 439 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:40:34.274999 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:40:34.287506 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-678fe433-cbb3-480c-8b07-247c0522ceb9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd2a240ceae7d424f8c2bda4b534eeb98'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:34.319599 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 867/1733] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 17:40:33 2022] DELETE /identity/v3/users/d2a240ceae7d424f8c2bda4b534eeb98 => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:40:34.323380 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2a4de2-d080-4ad1-95bc-faf87c5fc034 None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:34.324051 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2a4de2-d080-4ad1-95bc-faf87c5fc034 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:34.324585 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2a4de2-d080-4ad1-95bc-faf87c5fc034 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:34.397791 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-5f2a4de2-d080-4ad1-95bc-faf87c5fc034 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:40:34.401798 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-5f2a4de2-d080-4ad1-95bc-faf87c5fc034 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:34.659463 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 867/1734] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:40:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 340 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:40:34.712028 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678fe433-cbb3-480c-8b07-247c0522ceb9 admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:34.717902 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-678fe433-cbb3-480c-8b07-247c0522ceb9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:34.985233 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678fe433-cbb3-480c-8b07-247c0522ceb9 admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:35.017491 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95be09b-d27a-4391-a0f4-49d65d63c341 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43ad98a250a144099ccd18378aedd4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:35.018153 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95be09b-d27a-4391-a0f4-49d65d63c341 service neutron] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:35.025671 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95be09b-d27a-4391-a0f4-49d65d63c341 service neutron] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:35.025833 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95be09b-d27a-4391-a0f4-49d65d63c341 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:35.076485 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95be09b-d27a-4391-a0f4-49d65d63c341 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:35.078873 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95be09b-d27a-4391-a0f4-49d65d63c341 service neutron] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:35.103109 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 868/1735] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:40:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:40:35.929365 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a70f320-3fdb-4a77-b4f0-129ed3bca8bd None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:35.930745 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a70f320-3fdb-4a77-b4f0-129ed3bca8bd None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:35.931301 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a70f320-3fdb-4a77-b4f0-129ed3bca8bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:35.967263 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-1a70f320-3fdb-4a77-b4f0-129ed3bca8bd None None] MFA Rules not processed for user `43ad98a250a144099ccd18378aedd4b5`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:40:35.969183 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-1a70f320-3fdb-4a77-b4f0-129ed3bca8bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:35.978418 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 868/1736] 10.222.0.73 () {58 vars in 1015 bytes} [Wed Aug 24 17:40:35 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:40:36.088372 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95be09b-d27a-4391-a0f4-49d65d63c341 service neutron] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:36.098213 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a70f320-3fdb-4a77-b4f0-129ed3bca8bd None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:36.100494 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-1a70f320-3fdb-4a77-b4f0-129ed3bca8bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:36.107499 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95be09b-d27a-4391-a0f4-49d65d63c341 service neutron] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:36.127280 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b59401-b1e9-45a6-85f1-471c29e9bc22 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43ad98a250a144099ccd18378aedd4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:36.132624 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b59401-b1e9-45a6-85f1-471c29e9bc22 service neutron] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:36.134007 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b59401-b1e9-45a6-85f1-471c29e9bc22 service neutron] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:36.135397 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b59401-b1e9-45a6-85f1-471c29e9bc22 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:36.144118 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a70f320-3fdb-4a77-b4f0-129ed3bca8bd None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:36.153468 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b6eb99-8770-43e1-bf2e-92adb99dcc97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:36.154112 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b6eb99-8770-43e1-bf2e-92adb99dcc97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:36.154399 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b6eb99-8770-43e1-bf2e-92adb99dcc97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:36.155213 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b6eb99-8770-43e1-bf2e-92adb99dcc97 admin admin] PATH_INFO: `/v3/projects/a6550d4a026d481a9a81820075c5c4b5` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:36.156351 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b6eb99-8770-43e1-bf2e-92adb99dcc97 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a6550d4a026d481a9a81820075c5c4b5)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:36.159479 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b6eb99-8770-43e1-bf2e-92adb99dcc97 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:36.160099 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b59401-b1e9-45a6-85f1-471c29e9bc22 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:36.161838 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b59401-b1e9-45a6-85f1-471c29e9bc22 service neutron] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:36.175065 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:40:36.185422 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 869/1737] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:40:36 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:40:36.304191 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-e2b6eb99-8770-43e1-bf2e-92adb99dcc97 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a6550d4a026d481a9a81820075c5c4b5', 'request_id': 'req-e2b6eb99-8770-43e1-bf2e-92adb99dcc97'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:36.316989 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-e2b6eb99-8770-43e1-bf2e-92adb99dcc97 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a6550d4a026d481a9a81820075c5c4b5', 'request_id': 'req-e2b6eb99-8770-43e1-bf2e-92adb99dcc97'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:36.340943 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-e2b6eb99-8770-43e1-bf2e-92adb99dcc97 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a6550d4a026d481a9a81820075c5c4b5', 'request_id': 'req-e2b6eb99-8770-43e1-bf2e-92adb99dcc97'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:36.364404 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-e2b6eb99-8770-43e1-bf2e-92adb99dcc97 admin admin] The token cache is being invalidate because project a6550d4a026d481a9a81820075c5c4b5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81759) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:40:36.364830 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-e2b6eb99-8770-43e1-bf2e-92adb99dcc97 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:36.365790 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-e2b6eb99-8770-43e1-bf2e-92adb99dcc97 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:36.371521 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 869/1738] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 17:40:36 2022] DELETE /identity/v3/projects/a6550d4a026d481a9a81820075c5c4b5 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:40:36.374118 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7fce69-b244-4977-9d93-c7604ab6b4f5 None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:36.374806 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7fce69-b244-4977-9d93-c7604ab6b4f5 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:36.376280 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7fce69-b244-4977-9d93-c7604ab6b4f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:36.440807 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-ea7fce69-b244-4977-9d93-c7604ab6b4f5 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:40:36.446566 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-ea7fce69-b244-4977-9d93-c7604ab6b4f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:36.707599 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 870/1739] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:40:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 331 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:40:36.707782 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7529c026-384f-4b9d-841a-6fa08b9c144b None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:36.707782 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7529c026-384f-4b9d-841a-6fa08b9c144b None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:36.707782 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7529c026-384f-4b9d-841a-6fa08b9c144b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:36.751195 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-7529c026-384f-4b9d-841a-6fa08b9c144b None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:40:36.753050 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-7529c026-384f-4b9d-841a-6fa08b9c144b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:36.765058 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 870/1740] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:40:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:40:36.775218 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7fce69-b244-4977-9d93-c7604ab6b4f5 None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:36.804226 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7fce69-b244-4977-9d93-c7604ab6b4f5 None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:36.812997 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626ace1f-b65a-466f-aa3b-9ab93a1acd46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:36.814506 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626ace1f-b65a-466f-aa3b-9ab93a1acd46 admin admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:36.814948 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626ace1f-b65a-466f-aa3b-9ab93a1acd46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:36.815380 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626ace1f-b65a-466f-aa3b-9ab93a1acd46 admin admin] PATH_INFO: `/v3/domains` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:36.816331 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626ace1f-b65a-466f-aa3b-9ab93a1acd46 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:40:36.816887 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626ace1f-b65a-466f-aa3b-9ab93a1acd46 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:36.818888 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626ace1f-b65a-466f-aa3b-9ab93a1acd46 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:36.863301 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7529c026-384f-4b9d-841a-6fa08b9c144b None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:36.865784 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 871/1741] 10.222.0.45 () {58 vars in 1153 bytes} [Wed Aug 24 17:40:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:40:36.881093 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7529c026-384f-4b9d-841a-6fa08b9c144b None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:36.919968 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f7bb52-c57a-4cde-8df5-e66f53a10da1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:36.920794 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f7bb52-c57a-4cde-8df5-e66f53a10da1 admin admin] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:36.922808 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f7bb52-c57a-4cde-8df5-e66f53a10da1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:36.923769 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f7bb52-c57a-4cde-8df5-e66f53a10da1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:36.924628 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f7bb52-c57a-4cde-8df5-e66f53a10da1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:36.952313 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f7bb52-c57a-4cde-8df5-e66f53a10da1 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:37.060454 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 871/1742] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 17:40:36 2022] POST /identity/v3/projects => generated 381 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:40:37.065990 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626ace1f-b65a-466f-aa3b-9ab93a1acd46 admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:37.086993 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626ace1f-b65a-466f-aa3b-9ab93a1acd46 admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:37.333217 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0addebb-f11e-44c1-a5a9-02dfafe45cc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:37.333989 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0addebb-f11e-44c1-a5a9-02dfafe45cc6 admin admin] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:37.334276 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0addebb-f11e-44c1-a5a9-02dfafe45cc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:37.334549 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0addebb-f11e-44c1-a5a9-02dfafe45cc6 admin admin] PATH_INFO: `/v3/users` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:37.335219 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0addebb-f11e-44c1-a5a9-02dfafe45cc6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:37.355820 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0addebb-f11e-44c1-a5a9-02dfafe45cc6 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:37.385309 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=2 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:40:37.410517 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 872/1743] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 17:40:37 2022] POST /identity/v3/users => generated 291 bytes in 346 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:40:37.415147 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f7bb52-c57a-4cde-8df5-e66f53a10da1 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:37.430349 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f7bb52-c57a-4cde-8df5-e66f53a10da1 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:37.462800 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f64aeed-9823-4bed-a7de-8e13f8539c8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:37.463724 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f64aeed-9823-4bed-a7de-8e13f8539c8e admin admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:37.464204 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f64aeed-9823-4bed-a7de-8e13f8539c8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:37.465681 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f64aeed-9823-4bed-a7de-8e13f8539c8e admin admin] PATH_INFO: `/v3/roles` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:37.466575 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f64aeed-9823-4bed-a7de-8e13f8539c8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:37.468468 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f64aeed-9823-4bed-a7de-8e13f8539c8e admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:37.485262 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0addebb-f11e-44c1-a5a9-02dfafe45cc6 admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:37.493038 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 872/1744] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:40:37 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 17:40:37.499862 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0addebb-f11e-44c1-a5a9-02dfafe45cc6 admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:37.507008 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c6f588-654e-41c5-b6c8-9973cb7cbe19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:37.507714 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c6f588-654e-41c5-b6c8-9973cb7cbe19 admin admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:37.508122 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c6f588-654e-41c5-b6c8-9973cb7cbe19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:37.508469 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c6f588-654e-41c5-b6c8-9973cb7cbe19 admin admin] PATH_INFO: `/v3/roles` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:37.509265 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c6f588-654e-41c5-b6c8-9973cb7cbe19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:37.511892 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c6f588-654e-41c5-b6c8-9973cb7cbe19 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:37.588342 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f64aeed-9823-4bed-a7de-8e13f8539c8e admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:37.597233 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 873/1745] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:40:37 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:40:37.601892 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f64aeed-9823-4bed-a7de-8e13f8539c8e admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:37.615860 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6febedb1-b3ef-4c89-928a-b8a331f00ce3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:37.618616 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6febedb1-b3ef-4c89-928a-b8a331f00ce3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:37.621698 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6febedb1-b3ef-4c89-928a-b8a331f00ce3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:37.622258 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6febedb1-b3ef-4c89-928a-b8a331f00ce3 admin admin] PATH_INFO: `/v3/projects/54ebee69ad68418d8a87fe5ea7a73547/users/e4a6568bbaa6430c98fe1491770b0b5b/roles/02476a5b82eb4b188c18a5d706a3ef68` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:37.660240 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6febedb1-b3ef-4c89-928a-b8a331f00ce3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=54ebee69ad68418d8a87fe5ea7a73547, user_id=e4a6568bbaa6430c98fe1491770b0b5b, role_id=02476a5b82eb4b188c18a5d706a3ef68)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:37.669441 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6febedb1-b3ef-4c89-928a-b8a331f00ce3 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:37.720681 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0494abf0-226b-490e-9aa6-00dce6cd6549 None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:37.721714 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0494abf0-226b-490e-9aa6-00dce6cd6549 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:37.721714 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0494abf0-226b-490e-9aa6-00dce6cd6549 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:37.721985 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 873/1746] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 17:40:37 2022] PUT /identity/v3/projects/54ebee69ad68418d8a87fe5ea7a73547/users/e4a6568bbaa6430c98fe1491770b0b5b/roles/02476a5b82eb4b188c18a5d706a3ef68 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:40:37.732579 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6febedb1-b3ef-4c89-928a-b8a331f00ce3 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:37.737597 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-6febedb1-b3ef-4c89-928a-b8a331f00ce3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:37.785608 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-0494abf0-226b-490e-9aa6-00dce6cd6549 None None] MFA Rules not processed for user `e4a6568bbaa6430c98fe1491770b0b5b`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:40:37.791587 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-0494abf0-226b-490e-9aa6-00dce6cd6549 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:37.941142 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6febedb1-b3ef-4c89-928a-b8a331f00ce3 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:37.946636 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ddc92c-05e4-4acb-a3b2-c6aa287c8516 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:37.947451 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ddc92c-05e4-4acb-a3b2-c6aa287c8516 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:37.947798 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ddc92c-05e4-4acb-a3b2-c6aa287c8516 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:37.948195 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ddc92c-05e4-4acb-a3b2-c6aa287c8516 admin admin] PATH_INFO: `/v3/projects/a1316765024f429bae881e682da29c80` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:37.950508 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ddc92c-05e4-4acb-a3b2-c6aa287c8516 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a1316765024f429bae881e682da29c80)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:37.953015 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ddc92c-05e4-4acb-a3b2-c6aa287c8516 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:37.991690 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 874/1747] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:40:37 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:40:37.992694 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0494abf0-226b-490e-9aa6-00dce6cd6549 None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:38.023705 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0494abf0-226b-490e-9aa6-00dce6cd6549 None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:38.039713 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:40:38.040274 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29466f4d-a8c8-41e0-ac0f-39a27467964c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:38.040953 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29466f4d-a8c8-41e0-ac0f-39a27467964c admin admin] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:38.041237 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29466f4d-a8c8-41e0-ac0f-39a27467964c admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:38.041525 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29466f4d-a8c8-41e0-ac0f-39a27467964c admin admin] PATH_INFO: `/v3/projects` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:38.042172 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29466f4d-a8c8-41e0-ac0f-39a27467964c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:38.044499 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29466f4d-a8c8-41e0-ac0f-39a27467964c admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:38.053307 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-c0ddc92c-05e4-4acb-a3b2-c6aa287c8516 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a1316765024f429bae881e682da29c80', 'request_id': 'req-c0ddc92c-05e4-4acb-a3b2-c6aa287c8516'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:38.070015 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-c0ddc92c-05e4-4acb-a3b2-c6aa287c8516 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a1316765024f429bae881e682da29c80', 'request_id': 'req-c0ddc92c-05e4-4acb-a3b2-c6aa287c8516'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:38.083117 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-c0ddc92c-05e4-4acb-a3b2-c6aa287c8516 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a1316765024f429bae881e682da29c80', 'request_id': 'req-c0ddc92c-05e4-4acb-a3b2-c6aa287c8516'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:38.109844 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 875/1748] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 17:40:37 2022] POST /identity/v3/projects => generated 379 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:40:38.115603 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29466f4d-a8c8-41e0-ac0f-39a27467964c admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:38.117486 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-c0ddc92c-05e4-4acb-a3b2-c6aa287c8516 admin admin] The token cache is being invalidate because project a1316765024f429bae881e682da29c80 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81759) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:40:38.117791 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-c0ddc92c-05e4-4acb-a3b2-c6aa287c8516 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:38.119272 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-c0ddc92c-05e4-4acb-a3b2-c6aa287c8516 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:38.121709 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 874/1749] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 17:40:37 2022] DELETE /identity/v3/projects/a1316765024f429bae881e682da29c80 => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:40:38.129394 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33569cbe-83fa-4448-9e7b-ffd0dc8e798c None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:38.131294 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33569cbe-83fa-4448-9e7b-ffd0dc8e798c None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:38.132085 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33569cbe-83fa-4448-9e7b-ffd0dc8e798c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:38.176922 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-33569cbe-83fa-4448-9e7b-ffd0dc8e798c None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:40:38.178717 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-33569cbe-83fa-4448-9e7b-ffd0dc8e798c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:38.267984 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29466f4d-a8c8-41e0-ac0f-39a27467964c admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:38.276861 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c586533-e102-483d-b79b-ade4f75ae7ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:38.277687 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c586533-e102-483d-b79b-ade4f75ae7ce admin admin] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:38.278068 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c586533-e102-483d-b79b-ade4f75ae7ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:38.278473 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c586533-e102-483d-b79b-ade4f75ae7ce admin admin] PATH_INFO: `/v3/users` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:38.279471 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c586533-e102-483d-b79b-ade4f75ae7ce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:38.281432 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c586533-e102-483d-b79b-ade4f75ae7ce admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:38.305138 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 875/1750] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:40:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:40:38.314420 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfb8935-6d99-456d-88ab-ef65a39d61fb None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:38.314957 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfb8935-6d99-456d-88ab-ef65a39d61fb None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:38.315557 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfb8935-6d99-456d-88ab-ef65a39d61fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:38.318464 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 876/1751] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 17:40:38 2022] POST /identity/v3/users => generated 296 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:40:38.322183 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c586533-e102-483d-b79b-ade4f75ae7ce admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:38.325363 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-6c586533-e102-483d-b79b-ade4f75ae7ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:38.354961 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c586533-e102-483d-b79b-ade4f75ae7ce admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:38.357538 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-6dfb8935-6d99-456d-88ab-ef65a39d61fb None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:40:38.359349 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-6dfb8935-6d99-456d-88ab-ef65a39d61fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:38.362859 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257c383a-0fc6-4fc6-9f4e-3f27dbd7ea99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:38.363722 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257c383a-0fc6-4fc6-9f4e-3f27dbd7ea99 admin admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:38.364105 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257c383a-0fc6-4fc6-9f4e-3f27dbd7ea99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:38.364481 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257c383a-0fc6-4fc6-9f4e-3f27dbd7ea99 admin admin] PATH_INFO: `/v3/roles` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:38.366443 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257c383a-0fc6-4fc6-9f4e-3f27dbd7ea99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:38.368763 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257c383a-0fc6-4fc6-9f4e-3f27dbd7ea99 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:38.373720 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 876/1752] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:40:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:40:38.376706 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfb8935-6d99-456d-88ab-ef65a39d61fb None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:38.390716 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 877/1753] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:40:38 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 17:40:38.396033 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257c383a-0fc6-4fc6-9f4e-3f27dbd7ea99 admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:38.397155 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfb8935-6d99-456d-88ab-ef65a39d61fb None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:38.405041 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d688e5fc-6fae-45f2-b539-42adc80018e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:38.405755 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d688e5fc-6fae-45f2-b539-42adc80018e4 admin admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:38.406701 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d688e5fc-6fae-45f2-b539-42adc80018e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:38.407055 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d688e5fc-6fae-45f2-b539-42adc80018e4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:38.407814 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d688e5fc-6fae-45f2-b539-42adc80018e4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:40:38.408927 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d688e5fc-6fae-45f2-b539-42adc80018e4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:38.412047 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d688e5fc-6fae-45f2-b539-42adc80018e4 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:38.416181 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257c383a-0fc6-4fc6-9f4e-3f27dbd7ea99 admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:38.422096 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080e974f-c26c-46b5-89c7-c981e2fa0f59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:38.423016 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080e974f-c26c-46b5-89c7-c981e2fa0f59 admin admin] REQUEST_METHOD: `PUT` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:38.423404 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080e974f-c26c-46b5-89c7-c981e2fa0f59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:38.423750 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080e974f-c26c-46b5-89c7-c981e2fa0f59 admin admin] PATH_INFO: `/v3/projects/533620c1a10743248fdf05bc5d8b6002/users/456e9f30853c40eb864e8cbd1f05cb65/roles/e411ad9eb96e4e8d85b59b5b700ada7b` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:38.438442 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 877/1754] 10.222.0.45 () {58 vars in 1153 bytes} [Wed Aug 24 17:40:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:40:38.442173 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d688e5fc-6fae-45f2-b539-42adc80018e4 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:38.455679 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080e974f-c26c-46b5-89c7-c981e2fa0f59 admin admin] RBAC: Authorizing `identity:create_grant(project_id=533620c1a10743248fdf05bc5d8b6002, user_id=456e9f30853c40eb864e8cbd1f05cb65, role_id=e411ad9eb96e4e8d85b59b5b700ada7b)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:38.460305 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d688e5fc-6fae-45f2-b539-42adc80018e4 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:38.460425 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080e974f-c26c-46b5-89c7-c981e2fa0f59 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:38.468104 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae4c9bc-1567-4206-b490-dd2a80dfd1fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:38.469496 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae4c9bc-1567-4206-b490-dd2a80dfd1fc admin admin] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:38.469912 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae4c9bc-1567-4206-b490-dd2a80dfd1fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:38.470877 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae4c9bc-1567-4206-b490-dd2a80dfd1fc admin admin] PATH_INFO: `/v3/projects` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:38.471862 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae4c9bc-1567-4206-b490-dd2a80dfd1fc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:38.476976 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae4c9bc-1567-4206-b490-dd2a80dfd1fc admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:38.487599 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 878/1755] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 17:40:38 2022] PUT /identity/v3/projects/533620c1a10743248fdf05bc5d8b6002/users/456e9f30853c40eb864e8cbd1f05cb65/roles/e411ad9eb96e4e8d85b59b5b700ada7b => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:40:38.491183 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080e974f-c26c-46b5-89c7-c981e2fa0f59 admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:38.498744 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:INSERT=7,keystone:SELECT=93 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:40:38.555584 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 878/1756] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 17:40:38 2022] POST /identity/v3/projects => generated 387 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:40:38.559633 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae4c9bc-1567-4206-b490-dd2a80dfd1fc admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:38.648863 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080e974f-c26c-46b5-89c7-c981e2fa0f59 admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:38.715847 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae4c9bc-1567-4206-b490-dd2a80dfd1fc admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:38.740511 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0b04c9-4669-46ed-8457-17599d0b0c6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:38.741190 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0b04c9-4669-46ed-8457-17599d0b0c6b admin admin] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:38.741483 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0b04c9-4669-46ed-8457-17599d0b0c6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:38.741761 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0b04c9-4669-46ed-8457-17599d0b0c6b admin admin] PATH_INFO: `/v3/users` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:38.742403 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0b04c9-4669-46ed-8457-17599d0b0c6b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:38.744647 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8b0d95-b184-417e-ba17-a3010cc9ed9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:38.745403 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8b0d95-b184-417e-ba17-a3010cc9ed9c admin admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:38.746335 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8b0d95-b184-417e-ba17-a3010cc9ed9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:38.746937 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8b0d95-b184-417e-ba17-a3010cc9ed9c admin admin] PATH_INFO: `/v3/roles` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:38.747650 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8b0d95-b184-417e-ba17-a3010cc9ed9c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:38.750439 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0b04c9-4669-46ed-8457-17599d0b0c6b admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:38.750937 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8b0d95-b184-417e-ba17-a3010cc9ed9c admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:38.764406 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 879/1757] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:40:38 2022] GET /identity/v3/roles => generated 1420 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 17:40:38.766001 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4983bcd5-66b7-427d-a2d1-4c0b101862ef None None] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:38.766393 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4983bcd5-66b7-427d-a2d1-4c0b101862ef None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:38.766615 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4983bcd5-66b7-427d-a2d1-4c0b101862ef None None] PATH_INFO: `/` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:38.768736 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 880/1758] 10.222.0.45 () {54 vars in 921 bytes} [Wed Aug 24 17:40:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:40:38.770977 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4983bcd5-66b7-427d-a2d1-4c0b101862ef None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:38.793509 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4983bcd5-66b7-427d-a2d1-4c0b101862ef None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:38.797680 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 879/1759] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 17:40:38 2022] POST /identity/v3/users => generated 294 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:40:38.799443 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0f1b45-2c0d-405b-ad20-8adcbfb1d754 None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:38.799956 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0f1b45-2c0d-405b-ad20-8adcbfb1d754 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:38.800446 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0f1b45-2c0d-405b-ad20-8adcbfb1d754 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:38.804645 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dd47a4-6a65-4920-9d14-a144bd777dba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:38.805464 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dd47a4-6a65-4920-9d14-a144bd777dba admin admin] REQUEST_METHOD: `PUT` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:38.805600 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dd47a4-6a65-4920-9d14-a144bd777dba admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:38.805952 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dd47a4-6a65-4920-9d14-a144bd777dba admin admin] PATH_INFO: `/v3/projects/533620c1a10743248fdf05bc5d8b6002/users/456e9f30853c40eb864e8cbd1f05cb65/roles/e411ad9eb96e4e8d85b59b5b700ada7b` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:38.812447 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9dd47a4-6a65-4920-9d14-a144bd777dba admin admin] RBAC: Authorizing `identity:create_grant(project_id=533620c1a10743248fdf05bc5d8b6002, user_id=456e9f30853c40eb864e8cbd1f05cb65, role_id=e411ad9eb96e4e8d85b59b5b700ada7b)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:38.813503 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9dd47a4-6a65-4920-9d14-a144bd777dba admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:38.830650 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 881/1760] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 17:40:38 2022] PUT /identity/v3/projects/533620c1a10743248fdf05bc5d8b6002/users/456e9f30853c40eb864e8cbd1f05cb65/roles/e411ad9eb96e4e8d85b59b5b700ada7b => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:40:38.834146 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dd47a4-6a65-4920-9d14-a144bd777dba admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:38.856741 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-be0f1b45-2c0d-405b-ad20-8adcbfb1d754 None None] MFA Rules not processed for user `053b7ce6d6f649ed91a26e434dab805e`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:40:38.858131 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-be0f1b45-2c0d-405b-ad20-8adcbfb1d754 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:38.883560 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:40:39.005326 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dd47a4-6a65-4920-9d14-a144bd777dba admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:39.010302 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88078111-6afa-495f-be0b-056cfed7fd9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:39.011019 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88078111-6afa-495f-be0b-056cfed7fd9a admin admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:39.011564 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88078111-6afa-495f-be0b-056cfed7fd9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:39.011879 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88078111-6afa-495f-be0b-056cfed7fd9a admin admin] PATH_INFO: `/v3/roles` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:39.012654 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88078111-6afa-495f-be0b-056cfed7fd9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:39.019739 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88078111-6afa-495f-be0b-056cfed7fd9a admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:39.028622 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 880/1761] 10.222.0.45 () {58 vars in 1018 bytes} [Wed Aug 24 17:40:38 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:40:39.030702 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39efe32a-de29-450d-a6ec-fab8a25b9cf1 None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:39.031134 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39efe32a-de29-450d-a6ec-fab8a25b9cf1 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:39.031553 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39efe32a-de29-450d-a6ec-fab8a25b9cf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:39.043963 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 882/1762] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:40:38 2022] GET /identity/v3/roles => generated 1420 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:40:39.045300 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88078111-6afa-495f-be0b-056cfed7fd9a admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:39.058110 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88078111-6afa-495f-be0b-056cfed7fd9a admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:39.074427 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cdb10c-2162-4b22-90af-1e14c36df363 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:39.075256 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cdb10c-2162-4b22-90af-1e14c36df363 admin admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:39.075536 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cdb10c-2162-4b22-90af-1e14c36df363 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:39.075809 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cdb10c-2162-4b22-90af-1e14c36df363 admin admin] PATH_INFO: `/v3/roles` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:39.076457 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cdb10c-2162-4b22-90af-1e14c36df363 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:39.079243 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cdb10c-2162-4b22-90af-1e14c36df363 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:39.088768 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-39efe32a-de29-450d-a6ec-fab8a25b9cf1 None None] MFA Rules not processed for user `456e9f30853c40eb864e8cbd1f05cb65`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:40:39.090091 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-39efe32a-de29-450d-a6ec-fab8a25b9cf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:39.092525 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 883/1763] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:40:39 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:40:39.093473 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cdb10c-2162-4b22-90af-1e14c36df363 admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:39.096666 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-75cdb10c-2162-4b22-90af-1e14c36df363 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:39.112576 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cdb10c-2162-4b22-90af-1e14c36df363 admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:39.119617 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe06bbd-2cdb-466e-aed3-ad50b5598354 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '053b7ce6d6f649ed91a26e434dab805e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:39.120232 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe06bbd-2cdb-466e-aed3-ad50b5598354 service nova] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:39.120512 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe06bbd-2cdb-466e-aed3-ad50b5598354 service nova] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:39.120828 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe06bbd-2cdb-466e-aed3-ad50b5598354 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:39.137356 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe06bbd-2cdb-466e-aed3-ad50b5598354 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:39.139210 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe06bbd-2cdb-466e-aed3-ad50b5598354 service nova] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:39.153770 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 884/1764] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:40:39 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:40:39.154619 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe06bbd-2cdb-466e-aed3-ad50b5598354 service nova] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:39.161581 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:40:39.168612 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe06bbd-2cdb-466e-aed3-ad50b5598354 service nova] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:39.180041 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4c0276-e5e0-40a4-aaae-3f5b35d9f417 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:39.180414 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4c0276-e5e0-40a4-aaae-3f5b35d9f417 admin admin] REQUEST_METHOD: `PUT` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:39.180746 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4c0276-e5e0-40a4-aaae-3f5b35d9f417 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:39.181023 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4c0276-e5e0-40a4-aaae-3f5b35d9f417 admin admin] PATH_INFO: `/v3/projects/9f9a0fa1887547bfb86d395b2ba0e01d/users/62f3d3f849ee4f02987aebe501528309/roles/02476a5b82eb4b188c18a5d706a3ef68` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:39.202547 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4c0276-e5e0-40a4-aaae-3f5b35d9f417 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f9a0fa1887547bfb86d395b2ba0e01d, user_id=62f3d3f849ee4f02987aebe501528309, role_id=02476a5b82eb4b188c18a5d706a3ef68)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:39.205319 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4c0276-e5e0-40a4-aaae-3f5b35d9f417 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:39.226181 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 885/1765] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 17:40:39 2022] PUT /identity/v3/projects/9f9a0fa1887547bfb86d395b2ba0e01d/users/62f3d3f849ee4f02987aebe501528309/roles/02476a5b82eb4b188c18a5d706a3ef68 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:40:39.241459 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e82b975-1fee-43b6-95e9-ad7bd19c6b8c None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:39.241890 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e82b975-1fee-43b6-95e9-ad7bd19c6b8c None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:39.242727 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e82b975-1fee-43b6-95e9-ad7bd19c6b8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:39.249657 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 881/1766] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:40:39 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:40:39.259253 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39efe32a-de29-450d-a6ec-fab8a25b9cf1 None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:39.260746 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-39efe32a-de29-450d-a6ec-fab8a25b9cf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:39.281960 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-8e82b975-1fee-43b6-95e9-ad7bd19c6b8c None None] MFA Rules not processed for user `62f3d3f849ee4f02987aebe501528309`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:40:39.283653 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-8e82b975-1fee-43b6-95e9-ad7bd19c6b8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:39.332181 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39efe32a-de29-450d-a6ec-fab8a25b9cf1 None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:39.368205 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 886/1767] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:40:39 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:40:39.372714 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e82b975-1fee-43b6-95e9-ad7bd19c6b8c None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:39.394408 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:40:39.399590 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49931881-e4a1-422e-b6bb-53f353282519 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59be342541264942ad87f60ff4de6c8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:39.400277 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49931881-e4a1-422e-b6bb-53f353282519 service cinder] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:39.400739 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49931881-e4a1-422e-b6bb-53f353282519 service cinder] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:39.401087 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49931881-e4a1-422e-b6bb-53f353282519 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:39.448416 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e82b975-1fee-43b6-95e9-ad7bd19c6b8c None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:39.453073 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1804b61-9483-4236-845d-01a471286741 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:39.453674 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1804b61-9483-4236-845d-01a471286741 admin admin] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:39.453981 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1804b61-9483-4236-845d-01a471286741 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:39.454259 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1804b61-9483-4236-845d-01a471286741 admin admin] PATH_INFO: `/v3/projects` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:39.454906 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1804b61-9483-4236-845d-01a471286741 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:39.456819 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1804b61-9483-4236-845d-01a471286741 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:39.466502 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49931881-e4a1-422e-b6bb-53f353282519 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:39.467939 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49931881-e4a1-422e-b6bb-53f353282519 service cinder] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:39.477835 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 882/1768] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:40:39 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:40:39.487211 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 887/1769] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 17:40:39 2022] POST /identity/v3/projects => generated 385 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:40:39.489100 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49931881-e4a1-422e-b6bb-53f353282519 service cinder] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:39.499731 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49931881-e4a1-422e-b6bb-53f353282519 service cinder] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:39.559405 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1804b61-9483-4236-845d-01a471286741 admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:39.565339 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27b0789-8c77-4c6c-b99d-3b8959ddb737 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:39.565952 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27b0789-8c77-4c6c-b99d-3b8959ddb737 admin admin] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:39.566242 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27b0789-8c77-4c6c-b99d-3b8959ddb737 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:39.566502 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27b0789-8c77-4c6c-b99d-3b8959ddb737 admin admin] PATH_INFO: `/v3/users` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:39.567134 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27b0789-8c77-4c6c-b99d-3b8959ddb737 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:39.569075 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27b0789-8c77-4c6c-b99d-3b8959ddb737 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:39.596783 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 883/1770] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 17:40:39 2022] POST /identity/v3/users => generated 299 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:40:39.600711 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27b0789-8c77-4c6c-b99d-3b8959ddb737 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:39.611484 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27b0789-8c77-4c6c-b99d-3b8959ddb737 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:39.617829 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebe87b0-8e2f-49a7-8b94-8d3afa3bd0a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:39.619152 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebe87b0-8e2f-49a7-8b94-8d3afa3bd0a7 admin admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:39.619437 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebe87b0-8e2f-49a7-8b94-8d3afa3bd0a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:39.619712 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebe87b0-8e2f-49a7-8b94-8d3afa3bd0a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:39.620366 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebe87b0-8e2f-49a7-8b94-8d3afa3bd0a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:39.622113 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebe87b0-8e2f-49a7-8b94-8d3afa3bd0a7 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:39.622993 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1804b61-9483-4236-845d-01a471286741 admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:39.628371 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a98d90a-98b9-4f5e-b0cd-2d2110c06e55 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59be342541264942ad87f60ff4de6c8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:39.629083 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a98d90a-98b9-4f5e-b0cd-2d2110c06e55 service cinder] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:39.629430 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a98d90a-98b9-4f5e-b0cd-2d2110c06e55 service cinder] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:39.629929 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a98d90a-98b9-4f5e-b0cd-2d2110c06e55 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:39.631832 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-5a98d90a-98b9-4f5e-b0cd-2d2110c06e55 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:39.633821 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 884/1771] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:40:39 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 17:40:39.638497 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebe87b0-8e2f-49a7-8b94-8d3afa3bd0a7 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:39.644353 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:40:39.650897 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebe87b0-8e2f-49a7-8b94-8d3afa3bd0a7 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:39.656409 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc89b0f-c1a4-4515-ac29-59fde9f5a20f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:39.657054 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc89b0f-c1a4-4515-ac29-59fde9f5a20f admin admin] REQUEST_METHOD: `PUT` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:39.657408 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc89b0f-c1a4-4515-ac29-59fde9f5a20f admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:39.658219 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc89b0f-c1a4-4515-ac29-59fde9f5a20f admin admin] PATH_INFO: `/v3/projects/7287a1120e6146cead22d7d9e4b8345a/users/d964d1cdfe264ae5bce2aa2add0e0607/roles/e411ad9eb96e4e8d85b59b5b700ada7b` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:39.676458 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc89b0f-c1a4-4515-ac29-59fde9f5a20f admin admin] RBAC: Authorizing `identity:create_grant(project_id=7287a1120e6146cead22d7d9e4b8345a, user_id=d964d1cdfe264ae5bce2aa2add0e0607, role_id=e411ad9eb96e4e8d85b59b5b700ada7b)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:39.679457 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc89b0f-c1a4-4515-ac29-59fde9f5a20f admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:39.694768 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 885/1772] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 17:40:39 2022] PUT /identity/v3/projects/7287a1120e6146cead22d7d9e4b8345a/users/d964d1cdfe264ae5bce2aa2add0e0607/roles/e411ad9eb96e4e8d85b59b5b700ada7b => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:40:39.699246 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc89b0f-c1a4-4515-ac29-59fde9f5a20f admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:39.709945 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a98d90a-98b9-4f5e-b0cd-2d2110c06e55 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:39.711787 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a98d90a-98b9-4f5e-b0cd-2d2110c06e55 service cinder] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:39.722531 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 888/1773] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:40:39 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:40:39.780403 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:40:39.798609 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc89b0f-c1a4-4515-ac29-59fde9f5a20f admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:39.806529 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3b4210-4614-4f5e-8e31-a30e09fc3ec7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:39.807203 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3b4210-4614-4f5e-8e31-a30e09fc3ec7 admin admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:39.807503 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3b4210-4614-4f5e-8e31-a30e09fc3ec7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:39.807766 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3b4210-4614-4f5e-8e31-a30e09fc3ec7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:39.808419 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3b4210-4614-4f5e-8e31-a30e09fc3ec7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:39.810091 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3b4210-4614-4f5e-8e31-a30e09fc3ec7 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:39.822057 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 886/1774] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:40:39 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 17:40:39.826240 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a98d90a-98b9-4f5e-b0cd-2d2110c06e55 service cinder] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:39.839664 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a98d90a-98b9-4f5e-b0cd-2d2110c06e55 service cinder] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:39.850972 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe51a27-9926-4668-af15-23e6928bbeac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:39.851950 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe51a27-9926-4668-af15-23e6928bbeac admin admin] REQUEST_METHOD: `PUT` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:39.852396 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe51a27-9926-4668-af15-23e6928bbeac admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:39.852936 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe51a27-9926-4668-af15-23e6928bbeac admin admin] PATH_INFO: `/v3/projects/7287a1120e6146cead22d7d9e4b8345a/users/d964d1cdfe264ae5bce2aa2add0e0607/roles/e411ad9eb96e4e8d85b59b5b700ada7b` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:39.855263 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe51a27-9926-4668-af15-23e6928bbeac admin admin] RBAC: Authorizing `identity:create_grant(project_id=7287a1120e6146cead22d7d9e4b8345a, user_id=d964d1cdfe264ae5bce2aa2add0e0607, role_id=e411ad9eb96e4e8d85b59b5b700ada7b)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:39.858414 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe51a27-9926-4668-af15-23e6928bbeac admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:39.870168 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 889/1775] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 17:40:39 2022] PUT /identity/v3/projects/7287a1120e6146cead22d7d9e4b8345a/users/d964d1cdfe264ae5bce2aa2add0e0607/roles/e411ad9eb96e4e8d85b59b5b700ada7b => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:40:39.881659 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744cc660-3372-48bd-8879-496adda76082 None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:39.881952 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744cc660-3372-48bd-8879-496adda76082 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:39.882239 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744cc660-3372-48bd-8879-496adda76082 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:39.917818 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-744cc660-3372-48bd-8879-496adda76082 None None] MFA Rules not processed for user `d964d1cdfe264ae5bce2aa2add0e0607`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:40:39.919136 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-744cc660-3372-48bd-8879-496adda76082 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:39.986797 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 887/1776] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:40:39 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:40:39.997826 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe51a27-9926-4668-af15-23e6928bbeac admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:40.056828 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe51a27-9926-4668-af15-23e6928bbeac admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:40.060459 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa28ee0-92c6-4b1b-996e-5e29a3edd639 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59be342541264942ad87f60ff4de6c8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:40.061121 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa28ee0-92c6-4b1b-996e-5e29a3edd639 service cinder] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:40.061400 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa28ee0-92c6-4b1b-996e-5e29a3edd639 service cinder] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:40.061676 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa28ee0-92c6-4b1b-996e-5e29a3edd639 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:40.071965 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa28ee0-92c6-4b1b-996e-5e29a3edd639 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:40.073464 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa28ee0-92c6-4b1b-996e-5e29a3edd639 service cinder] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:40.086167 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 890/1777] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:40:39 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:40:40.164692 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744cc660-3372-48bd-8879-496adda76082 None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:40.233017 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744cc660-3372-48bd-8879-496adda76082 None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:40.245880 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68a5a32-6fc7-403c-9c13-e04998868e92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:40.246636 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68a5a32-6fc7-403c-9c13-e04998868e92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:40.246812 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68a5a32-6fc7-403c-9c13-e04998868e92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:40.247084 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68a5a32-6fc7-403c-9c13-e04998868e92 admin admin] PATH_INFO: `/v3/users/62f3d3f849ee4f02987aebe501528309` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:40.248238 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68a5a32-6fc7-403c-9c13-e04998868e92 admin admin] RBAC: Authorizing `identity:delete_user(user_id=62f3d3f849ee4f02987aebe501528309)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:40.250303 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68a5a32-6fc7-403c-9c13-e04998868e92 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:40.307104 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-e68a5a32-6fc7-403c-9c13-e04998868e92 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '62f3d3f849ee4f02987aebe501528309'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:40.308012 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-e68a5a32-6fc7-403c-9c13-e04998868e92 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '62f3d3f849ee4f02987aebe501528309'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:40.318026 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-e68a5a32-6fc7-403c-9c13-e04998868e92 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '62f3d3f849ee4f02987aebe501528309'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:40.318740 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-e68a5a32-6fc7-403c-9c13-e04998868e92 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '62f3d3f849ee4f02987aebe501528309'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:40.327688 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-e68a5a32-6fc7-403c-9c13-e04998868e92 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '62f3d3f849ee4f02987aebe501528309'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:40.343815 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:40:40.355388 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 888/1778] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 17:40:40 2022] DELETE /identity/v3/users/62f3d3f849ee4f02987aebe501528309 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:40:40.359849 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f0ac24-f90d-4eca-93ae-22b5d881a411 None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:40.360022 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f0ac24-f90d-4eca-93ae-22b5d881a411 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:40.360296 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f0ac24-f90d-4eca-93ae-22b5d881a411 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:40.379632 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-39f0ac24-f90d-4eca-93ae-22b5d881a411 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:40:40.380967 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-39f0ac24-f90d-4eca-93ae-22b5d881a411 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:40.440620 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 891/1779] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:40:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:40:40.450640 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68a5a32-6fc7-403c-9c13-e04998868e92 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:40.452120 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-e68a5a32-6fc7-403c-9c13-e04998868e92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:40.514811 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68a5a32-6fc7-403c-9c13-e04998868e92 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:40.521351 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07013706-5b66-4433-aa23-be24245e1b31 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43ad98a250a144099ccd18378aedd4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:40.521978 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07013706-5b66-4433-aa23-be24245e1b31 service neutron] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:40.522260 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07013706-5b66-4433-aa23-be24245e1b31 service neutron] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:40.522535 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07013706-5b66-4433-aa23-be24245e1b31 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:40.534639 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07013706-5b66-4433-aa23-be24245e1b31 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:40.536367 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07013706-5b66-4433-aa23-be24245e1b31 service neutron] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:40.547013 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 889/1780] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:40:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 17:40:40.828759 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f0ac24-f90d-4eca-93ae-22b5d881a411 None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:40.830416 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-39f0ac24-f90d-4eca-93ae-22b5d881a411 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:40.843788 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f0ac24-f90d-4eca-93ae-22b5d881a411 None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:40.848909 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da220aa-1c30-41fe-8e22-39ea64f406ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:40.849878 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da220aa-1c30-41fe-8e22-39ea64f406ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:40.850297 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da220aa-1c30-41fe-8e22-39ea64f406ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:40.850736 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da220aa-1c30-41fe-8e22-39ea64f406ff admin admin] PATH_INFO: `/v3/projects/9f9a0fa1887547bfb86d395b2ba0e01d` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:40.852212 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da220aa-1c30-41fe-8e22-39ea64f406ff admin admin] RBAC: Authorizing `identity:delete_project(project_id=9f9a0fa1887547bfb86d395b2ba0e01d)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:40.854421 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da220aa-1c30-41fe-8e22-39ea64f406ff admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:40.902036 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=97,keystone:INSERT=1,keystone:DELETE=2 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:40:40.917235 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-8da220aa-1c30-41fe-8e22-39ea64f406ff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9f9a0fa1887547bfb86d395b2ba0e01d', 'request_id': 'req-8da220aa-1c30-41fe-8e22-39ea64f406ff'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:40.939762 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-8da220aa-1c30-41fe-8e22-39ea64f406ff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9f9a0fa1887547bfb86d395b2ba0e01d', 'request_id': 'req-8da220aa-1c30-41fe-8e22-39ea64f406ff'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:40.948189 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-8da220aa-1c30-41fe-8e22-39ea64f406ff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9f9a0fa1887547bfb86d395b2ba0e01d', 'request_id': 'req-8da220aa-1c30-41fe-8e22-39ea64f406ff'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:40.968719 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-8da220aa-1c30-41fe-8e22-39ea64f406ff admin admin] The token cache is being invalidate because project 9f9a0fa1887547bfb86d395b2ba0e01d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81758) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:40:40.969176 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-8da220aa-1c30-41fe-8e22-39ea64f406ff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:40.985600 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-8da220aa-1c30-41fe-8e22-39ea64f406ff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:40.989996 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 892/1781] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 17:40:40 2022] DELETE /identity/v3/projects/9f9a0fa1887547bfb86d395b2ba0e01d => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:40:40.994330 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07013706-5b66-4433-aa23-be24245e1b31 service neutron] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:41.006210 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07013706-5b66-4433-aa23-be24245e1b31 service neutron] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:41.132515 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0b7c06-3a76-40c1-b30c-f7fa8ab37fe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:41.133291 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0b7c06-3a76-40c1-b30c-f7fa8ab37fe9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:41.133685 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0b7c06-3a76-40c1-b30c-f7fa8ab37fe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:41.134034 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0b7c06-3a76-40c1-b30c-f7fa8ab37fe9 admin admin] PATH_INFO: `/v3/users/d964d1cdfe264ae5bce2aa2add0e0607` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:41.135675 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0b7c06-3a76-40c1-b30c-f7fa8ab37fe9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d964d1cdfe264ae5bce2aa2add0e0607)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:41.139479 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0b7c06-3a76-40c1-b30c-f7fa8ab37fe9 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:41.297591 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-ae0b7c06-3a76-40c1-b30c-f7fa8ab37fe9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd964d1cdfe264ae5bce2aa2add0e0607'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:41.299078 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-ae0b7c06-3a76-40c1-b30c-f7fa8ab37fe9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd964d1cdfe264ae5bce2aa2add0e0607'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:41.310298 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-ae0b7c06-3a76-40c1-b30c-f7fa8ab37fe9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd964d1cdfe264ae5bce2aa2add0e0607'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:41.310719 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-ae0b7c06-3a76-40c1-b30c-f7fa8ab37fe9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd964d1cdfe264ae5bce2aa2add0e0607'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:41.321029 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-ae0b7c06-3a76-40c1-b30c-f7fa8ab37fe9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd964d1cdfe264ae5bce2aa2add0e0607'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:41.352863 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 890/1782] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 17:40:40 2022] DELETE /identity/v3/users/d964d1cdfe264ae5bce2aa2add0e0607 => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:40:41.853105 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da220aa-1c30-41fe-8e22-39ea64f406ff admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:41.855322 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-8da220aa-1c30-41fe-8e22-39ea64f406ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:41.876132 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da220aa-1c30-41fe-8e22-39ea64f406ff admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:41.966844 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec54dca-96c2-49af-8347-efeb5fc9caef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:41.967742 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec54dca-96c2-49af-8347-efeb5fc9caef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:41.968081 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec54dca-96c2-49af-8347-efeb5fc9caef admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:41.968487 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec54dca-96c2-49af-8347-efeb5fc9caef admin admin] PATH_INFO: `/v3/projects/7287a1120e6146cead22d7d9e4b8345a` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:41.971320 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec54dca-96c2-49af-8347-efeb5fc9caef admin admin] RBAC: Authorizing `identity:delete_project(project_id=7287a1120e6146cead22d7d9e4b8345a)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:41.974040 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec54dca-96c2-49af-8347-efeb5fc9caef admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:42.036822 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-dec54dca-96c2-49af-8347-efeb5fc9caef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7287a1120e6146cead22d7d9e4b8345a', 'request_id': 'req-dec54dca-96c2-49af-8347-efeb5fc9caef'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:42.062156 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-dec54dca-96c2-49af-8347-efeb5fc9caef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7287a1120e6146cead22d7d9e4b8345a', 'request_id': 'req-dec54dca-96c2-49af-8347-efeb5fc9caef'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:42.073061 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-dec54dca-96c2-49af-8347-efeb5fc9caef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7287a1120e6146cead22d7d9e4b8345a', 'request_id': 'req-dec54dca-96c2-49af-8347-efeb5fc9caef'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:42.102826 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-dec54dca-96c2-49af-8347-efeb5fc9caef admin admin] The token cache is being invalidate because project 7287a1120e6146cead22d7d9e4b8345a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81758) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:40:42.103126 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-dec54dca-96c2-49af-8347-efeb5fc9caef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:42.104530 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-dec54dca-96c2-49af-8347-efeb5fc9caef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:42.106842 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 893/1783] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 17:40:41 2022] DELETE /identity/v3/projects/7287a1120e6146cead22d7d9e4b8345a => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:40:42.111998 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82309aa5-1d46-4d0f-b540-46fad888b842 None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:42.112798 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82309aa5-1d46-4d0f-b540-46fad888b842 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:42.112798 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82309aa5-1d46-4d0f-b540-46fad888b842 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:42.142931 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-82309aa5-1d46-4d0f-b540-46fad888b842 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:40:42.144185 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-82309aa5-1d46-4d0f-b540-46fad888b842 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:42.155656 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:40:42.248590 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 891/1784] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:40:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:40:42.256226 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcc5a98-a85f-4e33-b186-768c30e18606 None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:42.256513 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcc5a98-a85f-4e33-b186-768c30e18606 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:42.256825 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcc5a98-a85f-4e33-b186-768c30e18606 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:42.309732 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-abcc5a98-a85f-4e33-b186-768c30e18606 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:40:42.311141 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-abcc5a98-a85f-4e33-b186-768c30e18606 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:42.324597 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 894/1785] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:40:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:40:42.329808 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82309aa5-1d46-4d0f-b540-46fad888b842 None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:42.343431 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82309aa5-1d46-4d0f-b540-46fad888b842 None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:42.354378 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d712f2-b895-4d05-8779-7dccba9fecc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:42.355429 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d712f2-b895-4d05-8779-7dccba9fecc8 admin admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:42.356063 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d712f2-b895-4d05-8779-7dccba9fecc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:42.357422 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d712f2-b895-4d05-8779-7dccba9fecc8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:42.358921 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d712f2-b895-4d05-8779-7dccba9fecc8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:40:42.360472 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d712f2-b895-4d05-8779-7dccba9fecc8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:42.365531 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d712f2-b895-4d05-8779-7dccba9fecc8 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:42.390838 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 892/1786] 10.222.0.45 () {58 vars in 1153 bytes} [Wed Aug 24 17:40:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:40:42.395084 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcc5a98-a85f-4e33-b186-768c30e18606 None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:42.416104 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcc5a98-a85f-4e33-b186-768c30e18606 None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:42.420007 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2386c514-ae65-42ab-a8b9-25afdbcd5b48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:42.420666 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2386c514-ae65-42ab-a8b9-25afdbcd5b48 admin admin] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:42.420945 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2386c514-ae65-42ab-a8b9-25afdbcd5b48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:42.421215 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2386c514-ae65-42ab-a8b9-25afdbcd5b48 admin admin] PATH_INFO: `/v3/projects` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:42.421863 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2386c514-ae65-42ab-a8b9-25afdbcd5b48 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:42.424872 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2386c514-ae65-42ab-a8b9-25afdbcd5b48 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:42.473538 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 895/1787] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 17:40:42 2022] POST /identity/v3/projects => generated 389 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:40:42.480191 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d712f2-b895-4d05-8779-7dccba9fecc8 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:42.496475 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d712f2-b895-4d05-8779-7dccba9fecc8 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:42.588953 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eddae3f-9f3c-4002-9f6c-013bdbe561f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:42.589674 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eddae3f-9f3c-4002-9f6c-013bdbe561f9 admin admin] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:42.590026 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eddae3f-9f3c-4002-9f6c-013bdbe561f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:42.590375 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eddae3f-9f3c-4002-9f6c-013bdbe561f9 admin admin] PATH_INFO: `/v3/users` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:42.591661 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eddae3f-9f3c-4002-9f6c-013bdbe561f9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:42.593828 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eddae3f-9f3c-4002-9f6c-013bdbe561f9 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:42.620482 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 893/1788] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 17:40:42 2022] POST /identity/v3/users => generated 295 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:40:42.624186 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2386c514-ae65-42ab-a8b9-25afdbcd5b48 admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:42.637867 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2386c514-ae65-42ab-a8b9-25afdbcd5b48 admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:42.645710 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a02be06-8535-4f4f-bc5b-a0bab9aeba1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:42.646392 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a02be06-8535-4f4f-bc5b-a0bab9aeba1a admin admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:42.646826 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a02be06-8535-4f4f-bc5b-a0bab9aeba1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:42.647120 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a02be06-8535-4f4f-bc5b-a0bab9aeba1a admin admin] PATH_INFO: `/v3/roles` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:42.647808 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a02be06-8535-4f4f-bc5b-a0bab9aeba1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:42.649446 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a02be06-8535-4f4f-bc5b-a0bab9aeba1a admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:42.660314 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 896/1789] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:40:42 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:40:42.664130 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eddae3f-9f3c-4002-9f6c-013bdbe561f9 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:42.689694 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eddae3f-9f3c-4002-9f6c-013bdbe561f9 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:42.724835 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fe2483-67e5-4cb4-8476-46ac4cb81e6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:42.724835 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fe2483-67e5-4cb4-8476-46ac4cb81e6c admin admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:42.725189 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fe2483-67e5-4cb4-8476-46ac4cb81e6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:42.725387 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fe2483-67e5-4cb4-8476-46ac4cb81e6c admin admin] PATH_INFO: `/v3/roles` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:42.726048 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fe2483-67e5-4cb4-8476-46ac4cb81e6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:42.728879 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fe2483-67e5-4cb4-8476-46ac4cb81e6c admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:42.760656 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 894/1790] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:40:42 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:40:42.764953 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a02be06-8535-4f4f-bc5b-a0bab9aeba1a admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:42.780408 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a02be06-8535-4f4f-bc5b-a0bab9aeba1a admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:42.789314 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb48482e-2084-49ad-bcae-71f6fa190bd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:42.790082 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb48482e-2084-49ad-bcae-71f6fa190bd9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:42.790433 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb48482e-2084-49ad-bcae-71f6fa190bd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:42.790863 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb48482e-2084-49ad-bcae-71f6fa190bd9 admin admin] PATH_INFO: `/v3/projects/972bb0394720412784dd3bd7588cb3ff/users/e8efa78ecac048d8a7665f58bc30471d/roles/02476a5b82eb4b188c18a5d706a3ef68` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:42.804879 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:DELETE=12,keystone:SELECT=85,keystone:INSERT=3 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:40:42.811471 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb48482e-2084-49ad-bcae-71f6fa190bd9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=972bb0394720412784dd3bd7588cb3ff, user_id=e8efa78ecac048d8a7665f58bc30471d, role_id=02476a5b82eb4b188c18a5d706a3ef68)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:42.815555 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb48482e-2084-49ad-bcae-71f6fa190bd9 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:42.846694 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 897/1791] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 17:40:42 2022] PUT /identity/v3/projects/972bb0394720412784dd3bd7588cb3ff/users/e8efa78ecac048d8a7665f58bc30471d/roles/02476a5b82eb4b188c18a5d706a3ef68 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:40:42.856575 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fe0caf-b233-4172-91a8-0eb42aee3cd6 None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:42.857072 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fe0caf-b233-4172-91a8-0eb42aee3cd6 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:42.857489 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fe0caf-b233-4172-91a8-0eb42aee3cd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:42.934019 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-e8fe0caf-b233-4172-91a8-0eb42aee3cd6 None None] MFA Rules not processed for user `e8efa78ecac048d8a7665f58bc30471d`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:40:42.936325 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-e8fe0caf-b233-4172-91a8-0eb42aee3cd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:43.085548 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 895/1792] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:40:42 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:40:43.089770 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb48482e-2084-49ad-bcae-71f6fa190bd9 admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:43.219304 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb48482e-2084-49ad-bcae-71f6fa190bd9 admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:43.229985 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e75c1f-c1c6-446f-9413-c0190d69bcfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:43.230665 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e75c1f-c1c6-446f-9413-c0190d69bcfa admin admin] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:43.230954 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e75c1f-c1c6-446f-9413-c0190d69bcfa admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:43.231636 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e75c1f-c1c6-446f-9413-c0190d69bcfa admin admin] PATH_INFO: `/v3/projects` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:43.232346 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e75c1f-c1c6-446f-9413-c0190d69bcfa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:43.234589 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e75c1f-c1c6-446f-9413-c0190d69bcfa admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:43.320339 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fe0caf-b233-4172-91a8-0eb42aee3cd6 None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:43.327020 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 898/1793] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 17:40:43 2022] POST /identity/v3/projects => generated 387 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:40:43.337713 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fe0caf-b233-4172-91a8-0eb42aee3cd6 None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:43.360669 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:40:43.547123 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec2730a-8b53-4518-b804-311ef9ca4a2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:43.548811 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec2730a-8b53-4518-b804-311ef9ca4a2d admin admin] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:43.549064 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec2730a-8b53-4518-b804-311ef9ca4a2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:43.549345 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec2730a-8b53-4518-b804-311ef9ca4a2d admin admin] PATH_INFO: `/v3/users` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:43.553346 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec2730a-8b53-4518-b804-311ef9ca4a2d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:43.561145 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec2730a-8b53-4518-b804-311ef9ca4a2d admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:43.641237 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 896/1794] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 17:40:43 2022] POST /identity/v3/users => generated 300 bytes in 321 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:40:43.645793 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e75c1f-c1c6-446f-9413-c0190d69bcfa admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:43.664735 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e75c1f-c1c6-446f-9413-c0190d69bcfa admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:43.674972 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f12a6c-0e63-4ad2-ae87-3184077f8426 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:43.677323 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f12a6c-0e63-4ad2-ae87-3184077f8426 admin admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:43.678722 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f12a6c-0e63-4ad2-ae87-3184077f8426 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:43.679446 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f12a6c-0e63-4ad2-ae87-3184077f8426 admin admin] PATH_INFO: `/v3/roles` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:43.700619 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f12a6c-0e63-4ad2-ae87-3184077f8426 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:43.707545 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f12a6c-0e63-4ad2-ae87-3184077f8426 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:43.764082 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 899/1795] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:40:43 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:40:43.766061 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec2730a-8b53-4518-b804-311ef9ca4a2d admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:43.782015 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec2730a-8b53-4518-b804-311ef9ca4a2d admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:43.798991 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38833f4-6e09-4af7-9042-2758d3b51142 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:43.802721 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38833f4-6e09-4af7-9042-2758d3b51142 admin admin] REQUEST_METHOD: `PUT` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:43.802818 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38833f4-6e09-4af7-9042-2758d3b51142 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:43.803510 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38833f4-6e09-4af7-9042-2758d3b51142 admin admin] PATH_INFO: `/v3/projects/a520c174201a44849d8b4b6a9cf10455/users/85880ecd3c7a4a1585e782b6dff895e4/roles/e411ad9eb96e4e8d85b59b5b700ada7b` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:43.853811 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38833f4-6e09-4af7-9042-2758d3b51142 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a520c174201a44849d8b4b6a9cf10455, user_id=85880ecd3c7a4a1585e782b6dff895e4, role_id=e411ad9eb96e4e8d85b59b5b700ada7b)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:43.857126 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38833f4-6e09-4af7-9042-2758d3b51142 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:43.883906 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 897/1796] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 17:40:43 2022] PUT /identity/v3/projects/a520c174201a44849d8b4b6a9cf10455/users/85880ecd3c7a4a1585e782b6dff895e4/roles/e411ad9eb96e4e8d85b59b5b700ada7b => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:40:43.884818 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f12a6c-0e63-4ad2-ae87-3184077f8426 admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:43.905322 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f12a6c-0e63-4ad2-ae87-3184077f8426 admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:44.005928 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b6b376-7e0b-4e0b-94a7-5e165f9665a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:44.006561 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b6b376-7e0b-4e0b-94a7-5e165f9665a4 admin admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:44.006843 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b6b376-7e0b-4e0b-94a7-5e165f9665a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:44.007120 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b6b376-7e0b-4e0b-94a7-5e165f9665a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:44.008003 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b6b376-7e0b-4e0b-94a7-5e165f9665a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:44.009662 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b6b376-7e0b-4e0b-94a7-5e165f9665a4 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:44.021614 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 900/1797] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:40:43 2022] GET /identity/v3/roles => generated 1420 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:40:44.026739 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38833f4-6e09-4af7-9042-2758d3b51142 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:44.050242 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38833f4-6e09-4af7-9042-2758d3b51142 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:44.072328 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8692c3a-4e1d-48b1-85de-1124f8c7889f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:44.073131 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8692c3a-4e1d-48b1-85de-1124f8c7889f admin admin] REQUEST_METHOD: `PUT` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:44.073409 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8692c3a-4e1d-48b1-85de-1124f8c7889f admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:44.073688 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8692c3a-4e1d-48b1-85de-1124f8c7889f admin admin] PATH_INFO: `/v3/projects/a520c174201a44849d8b4b6a9cf10455/users/85880ecd3c7a4a1585e782b6dff895e4/roles/e411ad9eb96e4e8d85b59b5b700ada7b` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:44.078002 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8692c3a-4e1d-48b1-85de-1124f8c7889f admin admin] RBAC: Authorizing `identity:create_grant(project_id=a520c174201a44849d8b4b6a9cf10455, user_id=85880ecd3c7a4a1585e782b6dff895e4, role_id=e411ad9eb96e4e8d85b59b5b700ada7b)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:44.084058 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8692c3a-4e1d-48b1-85de-1124f8c7889f admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:44.120527 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 898/1798] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 17:40:44 2022] PUT /identity/v3/projects/a520c174201a44849d8b4b6a9cf10455/users/85880ecd3c7a4a1585e782b6dff895e4/roles/e411ad9eb96e4e8d85b59b5b700ada7b => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:40:44.133843 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5d1d68-8456-47fc-bc0b-2e467ddacb01 None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:44.133843 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5d1d68-8456-47fc-bc0b-2e467ddacb01 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:44.134479 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5d1d68-8456-47fc-bc0b-2e467ddacb01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:44.207557 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-0d5d1d68-8456-47fc-bc0b-2e467ddacb01 None None] MFA Rules not processed for user `85880ecd3c7a4a1585e782b6dff895e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:40:44.209469 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-0d5d1d68-8456-47fc-bc0b-2e467ddacb01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:44.291544 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:40:44.334662 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 901/1799] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:40:44 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:40:44.342042 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8692c3a-4e1d-48b1-85de-1124f8c7889f admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:44.343921 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-f8692c3a-4e1d-48b1-85de-1124f8c7889f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:44.467335 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8692c3a-4e1d-48b1-85de-1124f8c7889f admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:44.472766 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d07a61-bfca-4912-aae5-edc16dacefd7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59be342541264942ad87f60ff4de6c8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:44.473674 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d07a61-bfca-4912-aae5-edc16dacefd7 service cinder] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:44.473956 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d07a61-bfca-4912-aae5-edc16dacefd7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:44.474229 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d07a61-bfca-4912-aae5-edc16dacefd7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:44.484591 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d07a61-bfca-4912-aae5-edc16dacefd7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:44.486595 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d07a61-bfca-4912-aae5-edc16dacefd7 service cinder] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:44.497151 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 899/1800] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:40:44 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:40:44.690587 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5d1d68-8456-47fc-bc0b-2e467ddacb01 None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:44.807407 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5d1d68-8456-47fc-bc0b-2e467ddacb01 None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:44.817623 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1405a99b-50c7-4373-bde6-9fdbfbf104b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:44.818386 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1405a99b-50c7-4373-bde6-9fdbfbf104b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:44.818708 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1405a99b-50c7-4373-bde6-9fdbfbf104b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:44.818939 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1405a99b-50c7-4373-bde6-9fdbfbf104b8 admin admin] PATH_INFO: `/v3/users/e8efa78ecac048d8a7665f58bc30471d` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:44.820291 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1405a99b-50c7-4373-bde6-9fdbfbf104b8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e8efa78ecac048d8a7665f58bc30471d)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:44.822363 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1405a99b-50c7-4373-bde6-9fdbfbf104b8 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:44.910914 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-1405a99b-50c7-4373-bde6-9fdbfbf104b8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e8efa78ecac048d8a7665f58bc30471d'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:44.912743 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-1405a99b-50c7-4373-bde6-9fdbfbf104b8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e8efa78ecac048d8a7665f58bc30471d'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:44.926194 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-1405a99b-50c7-4373-bde6-9fdbfbf104b8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e8efa78ecac048d8a7665f58bc30471d'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:44.927076 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-1405a99b-50c7-4373-bde6-9fdbfbf104b8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e8efa78ecac048d8a7665f58bc30471d'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:44.950839 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-1405a99b-50c7-4373-bde6-9fdbfbf104b8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e8efa78ecac048d8a7665f58bc30471d'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:44.972521 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 902/1801] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 17:40:44 2022] DELETE /identity/v3/users/e8efa78ecac048d8a7665f58bc30471d => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:40:44.984018 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da5f70a-ede9-4b1d-b094-187144a7c260 None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:44.984739 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da5f70a-ede9-4b1d-b094-187144a7c260 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:44.985234 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da5f70a-ede9-4b1d-b094-187144a7c260 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:45.016040 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-6da5f70a-ede9-4b1d-b094-187144a7c260 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:40:45.018470 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-6da5f70a-ede9-4b1d-b094-187144a7c260 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:45.149698 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 900/1802] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:40:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:40:45.162504 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1405a99b-50c7-4373-bde6-9fdbfbf104b8 admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:45.165050 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-1405a99b-50c7-4373-bde6-9fdbfbf104b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:45.277580 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1405a99b-50c7-4373-bde6-9fdbfbf104b8 admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:45.282413 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98e07cb-2c6e-4aea-bac3-b85cf56d9b4a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43ad98a250a144099ccd18378aedd4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:45.283571 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98e07cb-2c6e-4aea-bac3-b85cf56d9b4a service neutron] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:45.284079 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98e07cb-2c6e-4aea-bac3-b85cf56d9b4a service neutron] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:45.284578 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98e07cb-2c6e-4aea-bac3-b85cf56d9b4a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:45.296198 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98e07cb-2c6e-4aea-bac3-b85cf56d9b4a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:45.298016 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98e07cb-2c6e-4aea-bac3-b85cf56d9b4a service neutron] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:45.332272 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 903/1803] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:40:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:40:45.703759 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da5f70a-ede9-4b1d-b094-187144a7c260 None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:45.705277 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-6da5f70a-ede9-4b1d-b094-187144a7c260 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:45.719757 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da5f70a-ede9-4b1d-b094-187144a7c260 None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:45.727888 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcf5835-08a1-482e-88ae-71c5c1f00d80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:45.729199 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcf5835-08a1-482e-88ae-71c5c1f00d80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:45.729682 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcf5835-08a1-482e-88ae-71c5c1f00d80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:45.730133 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcf5835-08a1-482e-88ae-71c5c1f00d80 admin admin] PATH_INFO: `/v3/projects/972bb0394720412784dd3bd7588cb3ff` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:45.731771 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcf5835-08a1-482e-88ae-71c5c1f00d80 admin admin] RBAC: Authorizing `identity:delete_project(project_id=972bb0394720412784dd3bd7588cb3ff)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:45.733782 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcf5835-08a1-482e-88ae-71c5c1f00d80 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:45.735901 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:40:45.853715 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-8dcf5835-08a1-482e-88ae-71c5c1f00d80 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '972bb0394720412784dd3bd7588cb3ff', 'request_id': 'req-8dcf5835-08a1-482e-88ae-71c5c1f00d80'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:45.865790 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-8dcf5835-08a1-482e-88ae-71c5c1f00d80 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '972bb0394720412784dd3bd7588cb3ff', 'request_id': 'req-8dcf5835-08a1-482e-88ae-71c5c1f00d80'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:45.873230 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-8dcf5835-08a1-482e-88ae-71c5c1f00d80 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '972bb0394720412784dd3bd7588cb3ff', 'request_id': 'req-8dcf5835-08a1-482e-88ae-71c5c1f00d80'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:45.898629 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-8dcf5835-08a1-482e-88ae-71c5c1f00d80 admin admin] The token cache is being invalidate because project 972bb0394720412784dd3bd7588cb3ff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81759) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:40:45.899018 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-8dcf5835-08a1-482e-88ae-71c5c1f00d80 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:45.899992 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-8dcf5835-08a1-482e-88ae-71c5c1f00d80 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:45.902416 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 901/1804] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 17:40:45 2022] DELETE /identity/v3/projects/972bb0394720412784dd3bd7588cb3ff => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:40:45.906854 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98e07cb-2c6e-4aea-bac3-b85cf56d9b4a service neutron] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:45.926296 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98e07cb-2c6e-4aea-bac3-b85cf56d9b4a service neutron] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:45.937669 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:40:46.011766 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad13b1d-3abb-468c-9060-17ded23062eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:46.012423 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad13b1d-3abb-468c-9060-17ded23062eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:46.012728 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad13b1d-3abb-468c-9060-17ded23062eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:46.013000 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad13b1d-3abb-468c-9060-17ded23062eb admin admin] PATH_INFO: `/v3/users/85880ecd3c7a4a1585e782b6dff895e4` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:46.014145 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad13b1d-3abb-468c-9060-17ded23062eb admin admin] RBAC: Authorizing `identity:delete_user(user_id=85880ecd3c7a4a1585e782b6dff895e4)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:46.016082 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad13b1d-3abb-468c-9060-17ded23062eb admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:46.070909 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-bad13b1d-3abb-468c-9060-17ded23062eb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '85880ecd3c7a4a1585e782b6dff895e4'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:46.071694 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-bad13b1d-3abb-468c-9060-17ded23062eb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '85880ecd3c7a4a1585e782b6dff895e4'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:46.079206 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-bad13b1d-3abb-468c-9060-17ded23062eb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '85880ecd3c7a4a1585e782b6dff895e4'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:46.080916 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-bad13b1d-3abb-468c-9060-17ded23062eb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '85880ecd3c7a4a1585e782b6dff895e4'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:46.094902 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-bad13b1d-3abb-468c-9060-17ded23062eb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '85880ecd3c7a4a1585e782b6dff895e4'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:46.116290 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 904/1805] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 17:40:45 2022] DELETE /identity/v3/users/85880ecd3c7a4a1585e782b6dff895e4 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:40:46.423062 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcf5835-08a1-482e-88ae-71c5c1f00d80 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:46.424716 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-8dcf5835-08a1-482e-88ae-71c5c1f00d80 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:46.437612 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcf5835-08a1-482e-88ae-71c5c1f00d80 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:46.500576 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31b3e27-500c-486f-9855-e1616fac5175 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:46.501270 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31b3e27-500c-486f-9855-e1616fac5175 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:46.502080 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31b3e27-500c-486f-9855-e1616fac5175 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:46.502355 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31b3e27-500c-486f-9855-e1616fac5175 admin admin] PATH_INFO: `/v3/projects/a520c174201a44849d8b4b6a9cf10455` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:46.503550 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31b3e27-500c-486f-9855-e1616fac5175 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a520c174201a44849d8b4b6a9cf10455)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:46.505509 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31b3e27-500c-486f-9855-e1616fac5175 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:46.550638 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-d31b3e27-500c-486f-9855-e1616fac5175 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a520c174201a44849d8b4b6a9cf10455', 'request_id': 'req-d31b3e27-500c-486f-9855-e1616fac5175'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:46.555976 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-d31b3e27-500c-486f-9855-e1616fac5175 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a520c174201a44849d8b4b6a9cf10455', 'request_id': 'req-d31b3e27-500c-486f-9855-e1616fac5175'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:46.561930 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-d31b3e27-500c-486f-9855-e1616fac5175 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a520c174201a44849d8b4b6a9cf10455', 'request_id': 'req-d31b3e27-500c-486f-9855-e1616fac5175'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:46.574233 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-d31b3e27-500c-486f-9855-e1616fac5175 admin admin] The token cache is being invalidate because project a520c174201a44849d8b4b6a9cf10455 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81759) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:40:46.574520 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-d31b3e27-500c-486f-9855-e1616fac5175 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:46.575156 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-d31b3e27-500c-486f-9855-e1616fac5175 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:46.576640 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 902/1806] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 17:40:46 2022] DELETE /identity/v3/projects/a520c174201a44849d8b4b6a9cf10455 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:40:46.582056 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49aeb29e-258d-49ec-b748-fbbba939cbf8 None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:46.582523 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49aeb29e-258d-49ec-b748-fbbba939cbf8 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:46.582604 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49aeb29e-258d-49ec-b748-fbbba939cbf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:46.601920 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-49aeb29e-258d-49ec-b748-fbbba939cbf8 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:40:46.603124 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-49aeb29e-258d-49ec-b748-fbbba939cbf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:46.662570 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 905/1807] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:40:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:40:46.670232 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c3dad9-64ab-4f54-98a0-b97151e92a4c None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:46.670523 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c3dad9-64ab-4f54-98a0-b97151e92a4c None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:46.670803 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c3dad9-64ab-4f54-98a0-b97151e92a4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:46.691141 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-e9c3dad9-64ab-4f54-98a0-b97151e92a4c None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:40:46.692345 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-e9c3dad9-64ab-4f54-98a0-b97151e92a4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:46.700407 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 903/1808] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:40:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:40:46.704744 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49aeb29e-258d-49ec-b748-fbbba939cbf8 None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:46.714602 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49aeb29e-258d-49ec-b748-fbbba939cbf8 None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:46.719307 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db54a9a6-38ec-4ac1-bb93-5e4a1cb33c3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:46.719990 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db54a9a6-38ec-4ac1-bb93-5e4a1cb33c3a admin admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:46.720375 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db54a9a6-38ec-4ac1-bb93-5e4a1cb33c3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:46.720747 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db54a9a6-38ec-4ac1-bb93-5e4a1cb33c3a admin admin] PATH_INFO: `/v3/domains` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:46.721439 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db54a9a6-38ec-4ac1-bb93-5e4a1cb33c3a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:40:46.721754 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db54a9a6-38ec-4ac1-bb93-5e4a1cb33c3a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:46.723358 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db54a9a6-38ec-4ac1-bb93-5e4a1cb33c3a admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:46.737530 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 906/1809] 10.222.0.45 () {58 vars in 1153 bytes} [Wed Aug 24 17:40:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:40:46.747910 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c3dad9-64ab-4f54-98a0-b97151e92a4c None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:46.757995 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c3dad9-64ab-4f54-98a0-b97151e92a4c None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:46.762856 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26edf195-f9e3-44b2-99af-e98021b697a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:46.763458 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26edf195-f9e3-44b2-99af-e98021b697a8 admin admin] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:46.763739 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26edf195-f9e3-44b2-99af-e98021b697a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:46.764014 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26edf195-f9e3-44b2-99af-e98021b697a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:46.764674 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26edf195-f9e3-44b2-99af-e98021b697a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:46.766579 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26edf195-f9e3-44b2-99af-e98021b697a8 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:46.778195 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:40:46.795305 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 904/1810] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 17:40:46 2022] POST /identity/v3/projects => generated 375 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:40:46.800047 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db54a9a6-38ec-4ac1-bb93-5e4a1cb33c3a admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:46.810356 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db54a9a6-38ec-4ac1-bb93-5e4a1cb33c3a admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:46.820234 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:40:46.880367 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8238b28f-5ed0-41c5-bfdb-bc5c87daf44e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:46.881018 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8238b28f-5ed0-41c5-bfdb-bc5c87daf44e admin admin] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:46.881305 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8238b28f-5ed0-41c5-bfdb-bc5c87daf44e admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:46.881586 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8238b28f-5ed0-41c5-bfdb-bc5c87daf44e admin admin] PATH_INFO: `/v3/users` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:46.882265 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8238b28f-5ed0-41c5-bfdb-bc5c87daf44e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:46.884228 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8238b28f-5ed0-41c5-bfdb-bc5c87daf44e admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:46.913382 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 907/1811] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 17:40:46 2022] POST /identity/v3/users => generated 288 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:40:46.917821 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26edf195-f9e3-44b2-99af-e98021b697a8 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:46.929177 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26edf195-f9e3-44b2-99af-e98021b697a8 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:46.933692 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079491c2-77ea-49ae-8d97-d4e3d0fbda86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:46.934352 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079491c2-77ea-49ae-8d97-d4e3d0fbda86 admin admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:46.934635 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079491c2-77ea-49ae-8d97-d4e3d0fbda86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:46.934910 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079491c2-77ea-49ae-8d97-d4e3d0fbda86 admin admin] PATH_INFO: `/v3/roles` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:46.935560 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079491c2-77ea-49ae-8d97-d4e3d0fbda86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:46.937206 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079491c2-77ea-49ae-8d97-d4e3d0fbda86 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:46.948577 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 905/1812] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:40:46 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:40:46.952595 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8238b28f-5ed0-41c5-bfdb-bc5c87daf44e admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:46.963139 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8238b28f-5ed0-41c5-bfdb-bc5c87daf44e admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:46.969178 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd04bb8-d106-449f-a09e-ce276e9dbbea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:46.969880 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd04bb8-d106-449f-a09e-ce276e9dbbea admin admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:46.970073 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd04bb8-d106-449f-a09e-ce276e9dbbea admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:46.970387 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd04bb8-d106-449f-a09e-ce276e9dbbea admin admin] PATH_INFO: `/v3/roles` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:46.971646 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd04bb8-d106-449f-a09e-ce276e9dbbea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:46.973300 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd04bb8-d106-449f-a09e-ce276e9dbbea admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:46.984599 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 908/1813] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:40:46 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:40:46.989571 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079491c2-77ea-49ae-8d97-d4e3d0fbda86 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:47.001338 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079491c2-77ea-49ae-8d97-d4e3d0fbda86 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:47.007153 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29ae972-1b58-4c51-8e36-7c2af240c972 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:47.007922 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29ae972-1b58-4c51-8e36-7c2af240c972 admin admin] REQUEST_METHOD: `PUT` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:47.008201 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29ae972-1b58-4c51-8e36-7c2af240c972 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:47.008476 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29ae972-1b58-4c51-8e36-7c2af240c972 admin admin] PATH_INFO: `/v3/projects/d2347dd8f7804bd7a2e477d185b369bb/users/e5c0164c6a2d4584b4dcecf250d83306/roles/02476a5b82eb4b188c18a5d706a3ef68` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:47.033454 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29ae972-1b58-4c51-8e36-7c2af240c972 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d2347dd8f7804bd7a2e477d185b369bb, user_id=e5c0164c6a2d4584b4dcecf250d83306, role_id=02476a5b82eb4b188c18a5d706a3ef68)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:47.036202 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29ae972-1b58-4c51-8e36-7c2af240c972 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:47.050836 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 906/1814] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 17:40:46 2022] PUT /identity/v3/projects/d2347dd8f7804bd7a2e477d185b369bb/users/e5c0164c6a2d4584b4dcecf250d83306/roles/02476a5b82eb4b188c18a5d706a3ef68 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:40:47.061243 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8769c7b4-be50-4528-8b35-f466f343f870 None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:47.061536 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8769c7b4-be50-4528-8b35-f466f343f870 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:47.061808 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8769c7b4-be50-4528-8b35-f466f343f870 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:47.088461 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-8769c7b4-be50-4528-8b35-f466f343f870 None None] MFA Rules not processed for user `e5c0164c6a2d4584b4dcecf250d83306`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:40:47.089703 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-8769c7b4-be50-4528-8b35-f466f343f870 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:47.155251 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 909/1815] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:40:47 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:40:47.166764 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29ae972-1b58-4c51-8e36-7c2af240c972 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:47.168475 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-e29ae972-1b58-4c51-8e36-7c2af240c972 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:47.230142 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29ae972-1b58-4c51-8e36-7c2af240c972 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:47.234191 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6090bbd3-bcbe-4af5-a7c8-a687bb22a9b7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59be342541264942ad87f60ff4de6c8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:47.234789 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6090bbd3-bcbe-4af5-a7c8-a687bb22a9b7 service cinder] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:47.235067 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6090bbd3-bcbe-4af5-a7c8-a687bb22a9b7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:47.235337 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6090bbd3-bcbe-4af5-a7c8-a687bb22a9b7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:47.245144 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6090bbd3-bcbe-4af5-a7c8-a687bb22a9b7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:47.246565 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6090bbd3-bcbe-4af5-a7c8-a687bb22a9b7 service cinder] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:47.255449 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 907/1816] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:40:47 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:40:53.021346 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9252ff4c-e00a-4832-b6b0-012297ff0b73 None None] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:53.021883 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9252ff4c-e00a-4832-b6b0-012297ff0b73 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:53.021883 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9252ff4c-e00a-4832-b6b0-012297ff0b73 None None] PATH_INFO: `/` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:53.023657 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 910/1817] 10.222.0.45 () {54 vars in 921 bytes} [Wed Aug 24 17:40:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:40:53.036490 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1c599a-260a-49a8-b491-82761ada56c2 None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:53.037511 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1c599a-260a-49a8-b491-82761ada56c2 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:53.037953 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1c599a-260a-49a8-b491-82761ada56c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:53.059254 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-ca1c599a-260a-49a8-b491-82761ada56c2 None None] MFA Rules not processed for user `053b7ce6d6f649ed91a26e434dab805e`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:40:53.060712 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-ca1c599a-260a-49a8-b491-82761ada56c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:53.137845 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 908/1818] 10.222.0.45 () {58 vars in 1018 bytes} [Wed Aug 24 17:40:53 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:40:53.149705 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9252ff4c-e00a-4832-b6b0-012297ff0b73 None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:53.151352 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-9252ff4c-e00a-4832-b6b0-012297ff0b73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:53.169359 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9252ff4c-e00a-4832-b6b0-012297ff0b73 None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:53.175643 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f4a060-57a5-4ec2-9a16-274e7a0a27cb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '053b7ce6d6f649ed91a26e434dab805e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:53.176244 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f4a060-57a5-4ec2-9a16-274e7a0a27cb service nova] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:53.176526 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f4a060-57a5-4ec2-9a16-274e7a0a27cb service nova] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:53.176842 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f4a060-57a5-4ec2-9a16-274e7a0a27cb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:53.185403 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f4a060-57a5-4ec2-9a16-274e7a0a27cb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:53.186963 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f4a060-57a5-4ec2-9a16-274e7a0a27cb service nova] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:53.197005 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 911/1819] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:40:53 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:40:58.952222 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1c599a-260a-49a8-b491-82761ada56c2 None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:58.954090 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-ca1c599a-260a-49a8-b491-82761ada56c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:59.015055 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:40:59.035387 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1c599a-260a-49a8-b491-82761ada56c2 None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:59.042084 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93916d37-a973-481b-bf71-4224157ef049 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:59.042843 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93916d37-a973-481b-bf71-4224157ef049 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:59.043192 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93916d37-a973-481b-bf71-4224157ef049 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:59.043516 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93916d37-a973-481b-bf71-4224157ef049 admin admin] PATH_INFO: `/v3/users/2aefcf1d24db46d2a17f3953bc840094` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:59.045062 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93916d37-a973-481b-bf71-4224157ef049 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2aefcf1d24db46d2a17f3953bc840094)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:59.050207 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93916d37-a973-481b-bf71-4224157ef049 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:59.107057 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-93916d37-a973-481b-bf71-4224157ef049 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2aefcf1d24db46d2a17f3953bc840094'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:59.107708 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-93916d37-a973-481b-bf71-4224157ef049 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2aefcf1d24db46d2a17f3953bc840094'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:59.114798 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-93916d37-a973-481b-bf71-4224157ef049 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2aefcf1d24db46d2a17f3953bc840094'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:59.115487 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-93916d37-a973-481b-bf71-4224157ef049 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2aefcf1d24db46d2a17f3953bc840094'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:59.124634 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-93916d37-a973-481b-bf71-4224157ef049 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2aefcf1d24db46d2a17f3953bc840094'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:59.147982 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 909/1820] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 17:40:58 2022] DELETE /identity/v3/users/2aefcf1d24db46d2a17f3953bc840094 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:40:59.153686 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba98ae34-e25b-4405-bd5b-09e4eed1710b None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:59.154183 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba98ae34-e25b-4405-bd5b-09e4eed1710b None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:59.154428 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba98ae34-e25b-4405-bd5b-09e4eed1710b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:59.174578 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-ba98ae34-e25b-4405-bd5b-09e4eed1710b None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:40:59.175855 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-ba98ae34-e25b-4405-bd5b-09e4eed1710b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:59.235479 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 912/1821] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:40:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:40:59.248182 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93916d37-a973-481b-bf71-4224157ef049 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:59.249698 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-93916d37-a973-481b-bf71-4224157ef049 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:59.324898 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93916d37-a973-481b-bf71-4224157ef049 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:59.328448 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4215df38-bbf2-454d-8bdb-c57dd5a31c7e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43ad98a250a144099ccd18378aedd4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:59.329078 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4215df38-bbf2-454d-8bdb-c57dd5a31c7e service neutron] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:59.329355 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4215df38-bbf2-454d-8bdb-c57dd5a31c7e service neutron] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:59.329649 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4215df38-bbf2-454d-8bdb-c57dd5a31c7e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:59.338754 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4215df38-bbf2-454d-8bdb-c57dd5a31c7e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:59.340174 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4215df38-bbf2-454d-8bdb-c57dd5a31c7e service neutron] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:59.350141 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 910/1822] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:40:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:40:59.633120 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba98ae34-e25b-4405-bd5b-09e4eed1710b None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:59.634651 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-ba98ae34-e25b-4405-bd5b-09e4eed1710b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:59.652987 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba98ae34-e25b-4405-bd5b-09e4eed1710b None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:59.657810 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d94f73-9d41-4d33-bff6-a5aae36284d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:59.658356 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d94f73-9d41-4d33-bff6-a5aae36284d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:59.658636 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d94f73-9d41-4d33-bff6-a5aae36284d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:59.659000 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d94f73-9d41-4d33-bff6-a5aae36284d5 admin admin] PATH_INFO: `/v3/projects/fb57982c79744bc8836897fb056d74da` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:59.660085 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d94f73-9d41-4d33-bff6-a5aae36284d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fb57982c79744bc8836897fb056d74da)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:59.661918 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d94f73-9d41-4d33-bff6-a5aae36284d5 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:59.664328 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:40:59.709709 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-b4d94f73-9d41-4d33-bff6-a5aae36284d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fb57982c79744bc8836897fb056d74da', 'request_id': 'req-b4d94f73-9d41-4d33-bff6-a5aae36284d5'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:59.716645 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-b4d94f73-9d41-4d33-bff6-a5aae36284d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fb57982c79744bc8836897fb056d74da', 'request_id': 'req-b4d94f73-9d41-4d33-bff6-a5aae36284d5'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:59.722669 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-b4d94f73-9d41-4d33-bff6-a5aae36284d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fb57982c79744bc8836897fb056d74da', 'request_id': 'req-b4d94f73-9d41-4d33-bff6-a5aae36284d5'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:59.733031 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-b4d94f73-9d41-4d33-bff6-a5aae36284d5 admin admin] The token cache is being invalidate because project fb57982c79744bc8836897fb056d74da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81758) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:40:59.733292 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-b4d94f73-9d41-4d33-bff6-a5aae36284d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:59.733947 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-b4d94f73-9d41-4d33-bff6-a5aae36284d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:59.735624 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 913/1823] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 17:40:59 2022] DELETE /identity/v3/projects/fb57982c79744bc8836897fb056d74da => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:40:59.740526 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4215df38-bbf2-454d-8bdb-c57dd5a31c7e service neutron] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:59.750164 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4215df38-bbf2-454d-8bdb-c57dd5a31c7e service neutron] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:59.808309 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838c1a39-06ff-43df-8f0f-dbd110306ca6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:59.809088 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838c1a39-06ff-43df-8f0f-dbd110306ca6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:59.809369 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838c1a39-06ff-43df-8f0f-dbd110306ca6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:59.809639 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838c1a39-06ff-43df-8f0f-dbd110306ca6 admin admin] PATH_INFO: `/v3/users/019fa20317e74ef9ad1bb6ef28fd34fa` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:59.810806 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838c1a39-06ff-43df-8f0f-dbd110306ca6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=019fa20317e74ef9ad1bb6ef28fd34fa)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:59.814645 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838c1a39-06ff-43df-8f0f-dbd110306ca6 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:59.829405 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:40:59.863332 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-838c1a39-06ff-43df-8f0f-dbd110306ca6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '019fa20317e74ef9ad1bb6ef28fd34fa'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:59.863968 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-838c1a39-06ff-43df-8f0f-dbd110306ca6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '019fa20317e74ef9ad1bb6ef28fd34fa'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:59.870692 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-838c1a39-06ff-43df-8f0f-dbd110306ca6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '019fa20317e74ef9ad1bb6ef28fd34fa'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:59.871353 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-838c1a39-06ff-43df-8f0f-dbd110306ca6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '019fa20317e74ef9ad1bb6ef28fd34fa'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:59.878849 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-838c1a39-06ff-43df-8f0f-dbd110306ca6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '019fa20317e74ef9ad1bb6ef28fd34fa'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:59.896409 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 911/1824] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 17:40:59 2022] DELETE /identity/v3/users/019fa20317e74ef9ad1bb6ef28fd34fa => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:41:00.165231 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d94f73-9d41-4d33-bff6-a5aae36284d5 admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:00.166702 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-b4d94f73-9d41-4d33-bff6-a5aae36284d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:00.183750 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d94f73-9d41-4d33-bff6-a5aae36284d5 admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:00.251798 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3dcc60-109f-4fd5-886f-45bb339decc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:00.252455 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3dcc60-109f-4fd5-886f-45bb339decc8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:00.252766 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3dcc60-109f-4fd5-886f-45bb339decc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:00.253046 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3dcc60-109f-4fd5-886f-45bb339decc8 admin admin] PATH_INFO: `/v3/projects/afaae2a2b1d749deacbd8c2645bcb93f` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:00.254186 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3dcc60-109f-4fd5-886f-45bb339decc8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=afaae2a2b1d749deacbd8c2645bcb93f)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:00.256090 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3dcc60-109f-4fd5-886f-45bb339decc8 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:00.305325 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-bc3dcc60-109f-4fd5-886f-45bb339decc8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'afaae2a2b1d749deacbd8c2645bcb93f', 'request_id': 'req-bc3dcc60-109f-4fd5-886f-45bb339decc8'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:00.316321 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-bc3dcc60-109f-4fd5-886f-45bb339decc8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'afaae2a2b1d749deacbd8c2645bcb93f', 'request_id': 'req-bc3dcc60-109f-4fd5-886f-45bb339decc8'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:00.325586 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-bc3dcc60-109f-4fd5-886f-45bb339decc8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'afaae2a2b1d749deacbd8c2645bcb93f', 'request_id': 'req-bc3dcc60-109f-4fd5-886f-45bb339decc8'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:00.340692 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-bc3dcc60-109f-4fd5-886f-45bb339decc8 admin admin] The token cache is being invalidate because project afaae2a2b1d749deacbd8c2645bcb93f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81758) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:41:00.341072 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-bc3dcc60-109f-4fd5-886f-45bb339decc8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:00.341799 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-bc3dcc60-109f-4fd5-886f-45bb339decc8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:00.343814 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 914/1825] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 17:41:00 2022] DELETE /identity/v3/projects/afaae2a2b1d749deacbd8c2645bcb93f => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:41:00.349027 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71f55ab-dbc0-4b48-bbbb-7e1d386f5694 None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:00.349496 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71f55ab-dbc0-4b48-bbbb-7e1d386f5694 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:00.349572 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71f55ab-dbc0-4b48-bbbb-7e1d386f5694 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:00.369686 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-c71f55ab-dbc0-4b48-bbbb-7e1d386f5694 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:41:00.370945 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-c71f55ab-dbc0-4b48-bbbb-7e1d386f5694 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:00.444239 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 912/1826] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:41:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:41:00.446838 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76aa70c5-a96b-4ee7-80cc-ba6fb26a4ec6 None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:00.447149 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76aa70c5-a96b-4ee7-80cc-ba6fb26a4ec6 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:00.447899 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76aa70c5-a96b-4ee7-80cc-ba6fb26a4ec6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:00.469967 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-76aa70c5-a96b-4ee7-80cc-ba6fb26a4ec6 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:41:00.471266 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-76aa70c5-a96b-4ee7-80cc-ba6fb26a4ec6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:00.495667 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 915/1827] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:41:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:41:00.495891 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71f55ab-dbc0-4b48-bbbb-7e1d386f5694 None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:00.521475 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71f55ab-dbc0-4b48-bbbb-7e1d386f5694 None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:00.528296 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153e5753-2c72-42b8-a9c9-e03091193bbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:00.530348 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153e5753-2c72-42b8-a9c9-e03091193bbc admin admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:00.530644 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153e5753-2c72-42b8-a9c9-e03091193bbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:00.530922 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153e5753-2c72-42b8-a9c9-e03091193bbc admin admin] PATH_INFO: `/v3/domains` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:00.531590 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153e5753-2c72-42b8-a9c9-e03091193bbc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:41:00.531897 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153e5753-2c72-42b8-a9c9-e03091193bbc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:00.534088 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153e5753-2c72-42b8-a9c9-e03091193bbc admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:00.552071 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 913/1828] 10.222.0.45 () {58 vars in 1153 bytes} [Wed Aug 24 17:41:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:41:00.554064 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76aa70c5-a96b-4ee7-80cc-ba6fb26a4ec6 None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:00.567396 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76aa70c5-a96b-4ee7-80cc-ba6fb26a4ec6 None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:00.573308 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc82280-cad8-4c27-ad28-403b639a2245 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:00.574165 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc82280-cad8-4c27-ad28-403b639a2245 admin admin] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:00.574445 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc82280-cad8-4c27-ad28-403b639a2245 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:00.574713 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc82280-cad8-4c27-ad28-403b639a2245 admin admin] PATH_INFO: `/v3/projects` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:00.575339 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc82280-cad8-4c27-ad28-403b639a2245 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:00.577821 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc82280-cad8-4c27-ad28-403b639a2245 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:00.599868 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:41:00.616600 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 916/1829] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 17:41:00 2022] POST /identity/v3/projects => generated 365 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:41:00.620390 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153e5753-2c72-42b8-a9c9-e03091193bbc admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:00.631608 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153e5753-2c72-42b8-a9c9-e03091193bbc admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:00.728854 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fe73d9-e47f-4fa8-9c03-7120b8dea9b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:00.729489 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fe73d9-e47f-4fa8-9c03-7120b8dea9b8 admin admin] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:00.729831 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fe73d9-e47f-4fa8-9c03-7120b8dea9b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:00.730197 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fe73d9-e47f-4fa8-9c03-7120b8dea9b8 admin admin] PATH_INFO: `/v3/users` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:00.730923 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fe73d9-e47f-4fa8-9c03-7120b8dea9b8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:00.732950 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fe73d9-e47f-4fa8-9c03-7120b8dea9b8 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:00.760499 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 914/1830] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 17:41:00 2022] POST /identity/v3/users => generated 283 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:41:00.760657 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc82280-cad8-4c27-ad28-403b639a2245 admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:00.778740 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc82280-cad8-4c27-ad28-403b639a2245 admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:00.786538 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31c891c-be16-4b62-b096-008e4513d0d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:00.787166 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31c891c-be16-4b62-b096-008e4513d0d4 admin admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:00.787627 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31c891c-be16-4b62-b096-008e4513d0d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:00.787703 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31c891c-be16-4b62-b096-008e4513d0d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:00.788364 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31c891c-be16-4b62-b096-008e4513d0d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:00.790032 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31c891c-be16-4b62-b096-008e4513d0d4 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:00.811474 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 917/1831] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:41:00 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:41:00.812597 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fe73d9-e47f-4fa8-9c03-7120b8dea9b8 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:00.836333 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fe73d9-e47f-4fa8-9c03-7120b8dea9b8 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:00.841114 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4b06b2-4ec0-4b3c-b53f-33955e3941b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:00.841728 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4b06b2-4ec0-4b3c-b53f-33955e3941b3 admin admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:00.842015 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4b06b2-4ec0-4b3c-b53f-33955e3941b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:00.842290 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4b06b2-4ec0-4b3c-b53f-33955e3941b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:00.842936 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4b06b2-4ec0-4b3c-b53f-33955e3941b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:00.844580 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4b06b2-4ec0-4b3c-b53f-33955e3941b3 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:00.854095 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:41:00.870110 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 915/1832] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:41:00 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:41:00.871296 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31c891c-be16-4b62-b096-008e4513d0d4 admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:00.882988 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31c891c-be16-4b62-b096-008e4513d0d4 admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:00.888946 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497aee51-2342-4e35-b7a3-63a5bb75fb39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:00.889829 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497aee51-2342-4e35-b7a3-63a5bb75fb39 admin admin] REQUEST_METHOD: `PUT` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:00.890188 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497aee51-2342-4e35-b7a3-63a5bb75fb39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:00.890531 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497aee51-2342-4e35-b7a3-63a5bb75fb39 admin admin] PATH_INFO: `/v3/projects/59f0a6adaaf244a2965df3b0d6e5e95d/users/bbf2966ef211410d9f3399015ce05b86/roles/02476a5b82eb4b188c18a5d706a3ef68` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:00.914889 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497aee51-2342-4e35-b7a3-63a5bb75fb39 admin admin] RBAC: Authorizing `identity:create_grant(project_id=59f0a6adaaf244a2965df3b0d6e5e95d, user_id=bbf2966ef211410d9f3399015ce05b86, role_id=02476a5b82eb4b188c18a5d706a3ef68)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:00.918272 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497aee51-2342-4e35-b7a3-63a5bb75fb39 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:00.936029 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 918/1833] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 17:41:00 2022] PUT /identity/v3/projects/59f0a6adaaf244a2965df3b0d6e5e95d/users/bbf2966ef211410d9f3399015ce05b86/roles/02476a5b82eb4b188c18a5d706a3ef68 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:41:00.946300 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28bc880-3581-4f45-b1da-e576f88e3f90 None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:00.946590 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28bc880-3581-4f45-b1da-e576f88e3f90 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:00.946869 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28bc880-3581-4f45-b1da-e576f88e3f90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:00.977291 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-e28bc880-3581-4f45-b1da-e576f88e3f90 None None] MFA Rules not processed for user `bbf2966ef211410d9f3399015ce05b86`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:41:00.978540 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-e28bc880-3581-4f45-b1da-e576f88e3f90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:01.071463 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 916/1834] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:41:00 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:41:01.075650 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497aee51-2342-4e35-b7a3-63a5bb75fb39 admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:01.150426 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497aee51-2342-4e35-b7a3-63a5bb75fb39 admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:01.155986 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0f429e-c846-4460-a197-9206c99a6111 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:01.156642 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0f429e-c846-4460-a197-9206c99a6111 admin admin] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:01.156930 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0f429e-c846-4460-a197-9206c99a6111 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:01.157316 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0f429e-c846-4460-a197-9206c99a6111 admin admin] PATH_INFO: `/v3/projects` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:01.157904 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0f429e-c846-4460-a197-9206c99a6111 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:01.160714 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0f429e-c846-4460-a197-9206c99a6111 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:01.197450 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 919/1835] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 17:41:01 2022] POST /identity/v3/projects => generated 365 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:41:01.201136 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28bc880-3581-4f45-b1da-e576f88e3f90 None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:01.212651 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28bc880-3581-4f45-b1da-e576f88e3f90 None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:01.272867 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd49bfb-e3b9-4aca-ba62-80680a5d6303 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:01.273395 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd49bfb-e3b9-4aca-ba62-80680a5d6303 admin admin] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:01.273677 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd49bfb-e3b9-4aca-ba62-80680a5d6303 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:01.273952 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd49bfb-e3b9-4aca-ba62-80680a5d6303 admin admin] PATH_INFO: `/v3/users` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:01.274593 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd49bfb-e3b9-4aca-ba62-80680a5d6303 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:01.276728 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd49bfb-e3b9-4aca-ba62-80680a5d6303 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:01.299359 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 917/1836] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 17:41:01 2022] POST /identity/v3/users => generated 289 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:41:01.303176 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0f429e-c846-4460-a197-9206c99a6111 admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:01.314543 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0f429e-c846-4460-a197-9206c99a6111 admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:01.319590 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3593c33-d520-4861-b70c-68974e1e9815 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:01.320205 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3593c33-d520-4861-b70c-68974e1e9815 admin admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:01.320700 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3593c33-d520-4861-b70c-68974e1e9815 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:01.320981 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3593c33-d520-4861-b70c-68974e1e9815 admin admin] PATH_INFO: `/v3/roles` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:01.321653 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3593c33-d520-4861-b70c-68974e1e9815 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:01.323230 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3593c33-d520-4861-b70c-68974e1e9815 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:01.334877 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 920/1837] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:41:01 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:41:01.338940 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd49bfb-e3b9-4aca-ba62-80680a5d6303 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:01.349284 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd49bfb-e3b9-4aca-ba62-80680a5d6303 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:01.354491 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0eb895-67ac-4d6c-ac1a-7a9401527448 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:01.355151 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0eb895-67ac-4d6c-ac1a-7a9401527448 admin admin] REQUEST_METHOD: `PUT` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:01.355440 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0eb895-67ac-4d6c-ac1a-7a9401527448 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:01.355717 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0eb895-67ac-4d6c-ac1a-7a9401527448 admin admin] PATH_INFO: `/v3/projects/dc7407cc1aee45f5bb60119df71fb64c/users/2311da6de24e48999351c274db85e714/roles/e411ad9eb96e4e8d85b59b5b700ada7b` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:01.370794 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0eb895-67ac-4d6c-ac1a-7a9401527448 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc7407cc1aee45f5bb60119df71fb64c, user_id=2311da6de24e48999351c274db85e714, role_id=e411ad9eb96e4e8d85b59b5b700ada7b)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:01.373554 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0eb895-67ac-4d6c-ac1a-7a9401527448 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:01.387261 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 918/1838] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 17:41:01 2022] PUT /identity/v3/projects/dc7407cc1aee45f5bb60119df71fb64c/users/2311da6de24e48999351c274db85e714/roles/e411ad9eb96e4e8d85b59b5b700ada7b => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:41:01.391669 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3593c33-d520-4861-b70c-68974e1e9815 admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:01.402883 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3593c33-d520-4861-b70c-68974e1e9815 admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:01.468229 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e919007-fa6b-4c9d-88dd-4316c04d15b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:01.468875 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e919007-fa6b-4c9d-88dd-4316c04d15b0 admin admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:01.469482 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e919007-fa6b-4c9d-88dd-4316c04d15b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:01.469482 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e919007-fa6b-4c9d-88dd-4316c04d15b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:01.470084 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e919007-fa6b-4c9d-88dd-4316c04d15b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:01.471712 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e919007-fa6b-4c9d-88dd-4316c04d15b0 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:01.486325 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 921/1839] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:41:01 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:41:01.487283 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0eb895-67ac-4d6c-ac1a-7a9401527448 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:01.498944 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0eb895-67ac-4d6c-ac1a-7a9401527448 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:01.504401 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a541bd-aa0b-4683-9207-1f872b9b4735 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:01.505219 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a541bd-aa0b-4683-9207-1f872b9b4735 admin admin] REQUEST_METHOD: `PUT` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:01.505500 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a541bd-aa0b-4683-9207-1f872b9b4735 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:01.505778 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a541bd-aa0b-4683-9207-1f872b9b4735 admin admin] PATH_INFO: `/v3/projects/dc7407cc1aee45f5bb60119df71fb64c/users/2311da6de24e48999351c274db85e714/roles/e411ad9eb96e4e8d85b59b5b700ada7b` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:01.507692 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a541bd-aa0b-4683-9207-1f872b9b4735 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc7407cc1aee45f5bb60119df71fb64c, user_id=2311da6de24e48999351c274db85e714, role_id=e411ad9eb96e4e8d85b59b5b700ada7b)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:01.510367 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a541bd-aa0b-4683-9207-1f872b9b4735 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:01.521507 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 919/1840] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 17:41:01 2022] PUT /identity/v3/projects/dc7407cc1aee45f5bb60119df71fb64c/users/2311da6de24e48999351c274db85e714/roles/e411ad9eb96e4e8d85b59b5b700ada7b => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:41:01.532623 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42086071-472c-45d0-9955-57b0f8a8c70d None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:01.532954 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42086071-472c-45d0-9955-57b0f8a8c70d None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:01.533230 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42086071-472c-45d0-9955-57b0f8a8c70d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:01.561119 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-42086071-472c-45d0-9955-57b0f8a8c70d None None] MFA Rules not processed for user `2311da6de24e48999351c274db85e714`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:41:01.562310 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-42086071-472c-45d0-9955-57b0f8a8c70d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:01.579349 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:41:01.633647 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 922/1841] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:41:01 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:41:01.645800 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a541bd-aa0b-4683-9207-1f872b9b4735 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:01.647511 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-67a541bd-aa0b-4683-9207-1f872b9b4735 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:01.737659 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:41:01.771673 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a541bd-aa0b-4683-9207-1f872b9b4735 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:01.775543 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a99e82-026a-4d81-92f3-f68a88a3a026 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59be342541264942ad87f60ff4de6c8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:01.776929 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a99e82-026a-4d81-92f3-f68a88a3a026 service cinder] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:01.777291 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a99e82-026a-4d81-92f3-f68a88a3a026 service cinder] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:01.777532 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a99e82-026a-4d81-92f3-f68a88a3a026 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:01.791022 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a99e82-026a-4d81-92f3-f68a88a3a026 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:01.792523 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a99e82-026a-4d81-92f3-f68a88a3a026 service cinder] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:01.808385 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 920/1842] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:41:01 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:41:01.902119 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42086071-472c-45d0-9955-57b0f8a8c70d None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:01.913545 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42086071-472c-45d0-9955-57b0f8a8c70d None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:01.917179 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5b33ed-6b85-4f48-9fe9-cb2a4f563db7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59be342541264942ad87f60ff4de6c8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:01.917812 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5b33ed-6b85-4f48-9fe9-cb2a4f563db7 service cinder] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:01.918095 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5b33ed-6b85-4f48-9fe9-cb2a4f563db7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:01.918369 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5b33ed-6b85-4f48-9fe9-cb2a4f563db7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:01.983318 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5b33ed-6b85-4f48-9fe9-cb2a4f563db7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:01.985435 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5b33ed-6b85-4f48-9fe9-cb2a4f563db7 service cinder] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:01.995835 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 923/1843] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:41:01 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:41:03.271235 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a99e82-026a-4d81-92f3-f68a88a3a026 service cinder] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:03.272815 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-70a99e82-026a-4d81-92f3-f68a88a3a026 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:03.343501 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a99e82-026a-4d81-92f3-f68a88a3a026 service cinder] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:03.348642 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7719bc-74e7-4895-b396-58144a8cf9c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:03.349310 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7719bc-74e7-4895-b396-58144a8cf9c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:03.349612 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7719bc-74e7-4895-b396-58144a8cf9c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:03.349893 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7719bc-74e7-4895-b396-58144a8cf9c7 admin admin] PATH_INFO: `/v3/users/e5c0164c6a2d4584b4dcecf250d83306` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:03.351038 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7719bc-74e7-4895-b396-58144a8cf9c7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e5c0164c6a2d4584b4dcecf250d83306)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:03.353017 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7719bc-74e7-4895-b396-58144a8cf9c7 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:03.410303 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-6f7719bc-74e7-4895-b396-58144a8cf9c7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e5c0164c6a2d4584b4dcecf250d83306'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:03.410996 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-6f7719bc-74e7-4895-b396-58144a8cf9c7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e5c0164c6a2d4584b4dcecf250d83306'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:03.419006 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-6f7719bc-74e7-4895-b396-58144a8cf9c7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e5c0164c6a2d4584b4dcecf250d83306'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:03.419681 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-6f7719bc-74e7-4895-b396-58144a8cf9c7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e5c0164c6a2d4584b4dcecf250d83306'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:03.427883 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-6f7719bc-74e7-4895-b396-58144a8cf9c7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e5c0164c6a2d4584b4dcecf250d83306'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:03.448193 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 921/1844] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 17:41:03 2022] DELETE /identity/v3/users/e5c0164c6a2d4584b4dcecf250d83306 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:41:03.452703 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd59d633-c05b-412a-9882-6289077ed14e None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:03.453173 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd59d633-c05b-412a-9882-6289077ed14e None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:03.453541 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd59d633-c05b-412a-9882-6289077ed14e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:03.472636 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-cd59d633-c05b-412a-9882-6289077ed14e None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:41:03.473892 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-cd59d633-c05b-412a-9882-6289077ed14e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:03.534577 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 924/1845] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:41:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:41:03.544224 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7719bc-74e7-4895-b396-58144a8cf9c7 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:03.545675 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-6f7719bc-74e7-4895-b396-58144a8cf9c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:03.612443 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7719bc-74e7-4895-b396-58144a8cf9c7 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:03.621611 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67471c8-531f-45b6-bc76-8b45c3492b8f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43ad98a250a144099ccd18378aedd4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:03.622240 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67471c8-531f-45b6-bc76-8b45c3492b8f service neutron] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:03.622518 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67471c8-531f-45b6-bc76-8b45c3492b8f service neutron] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:03.622789 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67471c8-531f-45b6-bc76-8b45c3492b8f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:03.632425 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67471c8-531f-45b6-bc76-8b45c3492b8f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:03.633988 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67471c8-531f-45b6-bc76-8b45c3492b8f service neutron] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:03.644705 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 922/1846] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:41:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:41:03.913421 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd59d633-c05b-412a-9882-6289077ed14e None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:03.915515 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-cd59d633-c05b-412a-9882-6289077ed14e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:03.938886 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd59d633-c05b-412a-9882-6289077ed14e None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:03.943889 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9efe2f-e21a-45ce-a4ad-a124531d31ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:03.944529 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9efe2f-e21a-45ce-a4ad-a124531d31ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:03.945115 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9efe2f-e21a-45ce-a4ad-a124531d31ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:03.945404 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9efe2f-e21a-45ce-a4ad-a124531d31ce admin admin] PATH_INFO: `/v3/projects/d2347dd8f7804bd7a2e477d185b369bb` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:03.946546 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9efe2f-e21a-45ce-a4ad-a124531d31ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=d2347dd8f7804bd7a2e477d185b369bb)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:03.948453 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9efe2f-e21a-45ce-a4ad-a124531d31ce admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:03.999458 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:41:04.000942 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-1b9efe2f-e21a-45ce-a4ad-a124531d31ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd2347dd8f7804bd7a2e477d185b369bb', 'request_id': 'req-1b9efe2f-e21a-45ce-a4ad-a124531d31ce'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:04.009427 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-1b9efe2f-e21a-45ce-a4ad-a124531d31ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd2347dd8f7804bd7a2e477d185b369bb', 'request_id': 'req-1b9efe2f-e21a-45ce-a4ad-a124531d31ce'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:04.015935 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-1b9efe2f-e21a-45ce-a4ad-a124531d31ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd2347dd8f7804bd7a2e477d185b369bb', 'request_id': 'req-1b9efe2f-e21a-45ce-a4ad-a124531d31ce'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:04.026742 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-1b9efe2f-e21a-45ce-a4ad-a124531d31ce admin admin] The token cache is being invalidate because project d2347dd8f7804bd7a2e477d185b369bb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81758) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:41:04.027033 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-1b9efe2f-e21a-45ce-a4ad-a124531d31ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:04.027670 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-1b9efe2f-e21a-45ce-a4ad-a124531d31ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:04.029205 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 925/1847] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 17:41:03 2022] DELETE /identity/v3/projects/d2347dd8f7804bd7a2e477d185b369bb => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:41:04.035393 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb3ab2d-4658-4104-84af-207f30ae3565 None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:04.035683 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb3ab2d-4658-4104-84af-207f30ae3565 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:04.035962 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb3ab2d-4658-4104-84af-207f30ae3565 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:04.044858 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:41:04.082222 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-7fb3ab2d-4658-4104-84af-207f30ae3565 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:41:04.083665 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-7fb3ab2d-4658-4104-84af-207f30ae3565 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:04.166868 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 923/1848] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:41:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:41:04.170907 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0175d4c2-3939-4881-9a8a-57321403048a None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:04.171195 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0175d4c2-3939-4881-9a8a-57321403048a None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:04.171470 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0175d4c2-3939-4881-9a8a-57321403048a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:04.192299 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-0175d4c2-3939-4881-9a8a-57321403048a None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:41:04.193643 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-0175d4c2-3939-4881-9a8a-57321403048a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:04.202404 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 926/1849] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:41:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:41:04.207152 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb3ab2d-4658-4104-84af-207f30ae3565 None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:04.217302 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb3ab2d-4658-4104-84af-207f30ae3565 None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:04.224491 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cdb2c0-8784-4778-840d-6d2be1887247 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:04.225216 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cdb2c0-8784-4778-840d-6d2be1887247 admin admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:04.225599 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cdb2c0-8784-4778-840d-6d2be1887247 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:04.225953 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cdb2c0-8784-4778-840d-6d2be1887247 admin admin] PATH_INFO: `/v3/domains` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:04.226874 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cdb2c0-8784-4778-840d-6d2be1887247 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:41:04.227449 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cdb2c0-8784-4778-840d-6d2be1887247 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:04.229466 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cdb2c0-8784-4778-840d-6d2be1887247 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:04.243873 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 924/1850] 10.222.0.45 () {58 vars in 1153 bytes} [Wed Aug 24 17:41:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:41:04.249305 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cdb2c0-8784-4778-840d-6d2be1887247 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:04.249460 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0175d4c2-3939-4881-9a8a-57321403048a None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:04.250143 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-d1cdb2c0-8784-4778-840d-6d2be1887247 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:04.263263 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0175d4c2-3939-4881-9a8a-57321403048a None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:04.269054 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cdb2c0-8784-4778-840d-6d2be1887247 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:04.269256 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b898c5-f29f-4278-b0ec-a0438488a208 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:04.269935 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b898c5-f29f-4278-b0ec-a0438488a208 admin admin] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:04.270220 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b898c5-f29f-4278-b0ec-a0438488a208 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:04.270486 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b898c5-f29f-4278-b0ec-a0438488a208 admin admin] PATH_INFO: `/v3/projects` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:04.271481 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b898c5-f29f-4278-b0ec-a0438488a208 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:04.273424 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b898c5-f29f-4278-b0ec-a0438488a208 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:04.275520 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c08048-9cac-4c58-8b62-177b644283f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:04.276315 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c08048-9cac-4c58-8b62-177b644283f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:04.276791 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c08048-9cac-4c58-8b62-177b644283f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:04.277090 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c08048-9cac-4c58-8b62-177b644283f8 admin admin] PATH_INFO: `/v3/users/e4a6568bbaa6430c98fe1491770b0b5b` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:04.278226 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c08048-9cac-4c58-8b62-177b644283f8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e4a6568bbaa6430c98fe1491770b0b5b)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:04.280520 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c08048-9cac-4c58-8b62-177b644283f8 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:04.312188 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 927/1851] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 17:41:04 2022] POST /identity/v3/projects => generated 383 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:41:04.317936 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b898c5-f29f-4278-b0ec-a0438488a208 admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:04.363822 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-20c08048-9cac-4c58-8b62-177b644283f8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e4a6568bbaa6430c98fe1491770b0b5b'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:04.364754 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-20c08048-9cac-4c58-8b62-177b644283f8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e4a6568bbaa6430c98fe1491770b0b5b'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:04.372267 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-20c08048-9cac-4c58-8b62-177b644283f8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e4a6568bbaa6430c98fe1491770b0b5b'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:04.373068 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-20c08048-9cac-4c58-8b62-177b644283f8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e4a6568bbaa6430c98fe1491770b0b5b'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:04.384776 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-20c08048-9cac-4c58-8b62-177b644283f8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e4a6568bbaa6430c98fe1491770b0b5b'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:04.414044 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 925/1852] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 17:41:04 2022] DELETE /identity/v3/users/e4a6568bbaa6430c98fe1491770b0b5b => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:41:04.418379 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b898c5-f29f-4278-b0ec-a0438488a208 admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:04.419362 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac42afe-f3cf-43d8-aaaf-a77f971bb4eb None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:04.430475 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac42afe-f3cf-43d8-aaaf-a77f971bb4eb None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:04.430901 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac42afe-f3cf-43d8-aaaf-a77f971bb4eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:04.458041 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-2ac42afe-f3cf-43d8-aaaf-a77f971bb4eb None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:41:04.459020 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-2ac42afe-f3cf-43d8-aaaf-a77f971bb4eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:04.518392 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ca29dc-032a-436c-9a5f-8cdef56d9f5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:04.519688 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ca29dc-032a-436c-9a5f-8cdef56d9f5f admin admin] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:04.520788 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ca29dc-032a-436c-9a5f-8cdef56d9f5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:04.521528 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ca29dc-032a-436c-9a5f-8cdef56d9f5f admin admin] PATH_INFO: `/v3/users` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:04.522475 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ca29dc-032a-436c-9a5f-8cdef56d9f5f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:04.525271 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ca29dc-032a-436c-9a5f-8cdef56d9f5f admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:04.551933 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 926/1853] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:41:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:41:04.558680 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 928/1854] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 17:41:04 2022] POST /identity/v3/users => generated 292 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:41:04.561612 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac42afe-f3cf-43d8-aaaf-a77f971bb4eb None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:04.563109 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-2ac42afe-f3cf-43d8-aaaf-a77f971bb4eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:04.563515 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ca29dc-032a-436c-9a5f-8cdef56d9f5f admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:04.565825 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-d0ca29dc-032a-436c-9a5f-8cdef56d9f5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:04.573553 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:41:04.578111 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac42afe-f3cf-43d8-aaaf-a77f971bb4eb None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:04.585536 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a7624a-709c-40fa-855f-357e57dd620e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:04.586199 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a7624a-709c-40fa-855f-357e57dd620e admin admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:04.586488 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a7624a-709c-40fa-855f-357e57dd620e admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:04.586765 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a7624a-709c-40fa-855f-357e57dd620e admin admin] PATH_INFO: `/v3/roles` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:04.587456 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a7624a-709c-40fa-855f-357e57dd620e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:04.589589 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a7624a-709c-40fa-855f-357e57dd620e admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:04.603907 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 927/1855] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:41:04 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:41:04.604871 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a7624a-709c-40fa-855f-357e57dd620e admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:04.618941 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a7624a-709c-40fa-855f-357e57dd620e admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:04.626334 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:41:04.628063 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c749fefa-3b4b-437d-85d8-6818b1a1b6da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:04.629328 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c749fefa-3b4b-437d-85d8-6818b1a1b6da admin admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:04.629999 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c749fefa-3b4b-437d-85d8-6818b1a1b6da admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:04.631048 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c749fefa-3b4b-437d-85d8-6818b1a1b6da admin admin] PATH_INFO: `/v3/roles` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:04.632030 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c749fefa-3b4b-437d-85d8-6818b1a1b6da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:04.634830 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c749fefa-3b4b-437d-85d8-6818b1a1b6da admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:04.642960 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ca29dc-032a-436c-9a5f-8cdef56d9f5f admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:04.650288 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc36adef-859a-47ad-918f-bbbe04f13c8f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43ad98a250a144099ccd18378aedd4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:04.650288 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc36adef-859a-47ad-918f-bbbe04f13c8f service neutron] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:04.650288 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc36adef-859a-47ad-918f-bbbe04f13c8f service neutron] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:04.650288 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc36adef-859a-47ad-918f-bbbe04f13c8f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:04.656613 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 928/1856] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:41:04 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:41:04.658841 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c749fefa-3b4b-437d-85d8-6818b1a1b6da admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:04.661482 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc36adef-859a-47ad-918f-bbbe04f13c8f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:04.663412 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc36adef-859a-47ad-918f-bbbe04f13c8f service neutron] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:04.668979 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c749fefa-3b4b-437d-85d8-6818b1a1b6da admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:04.674242 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bbeb61-5e95-4dc0-9172-2b49268fcbbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:04.674902 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bbeb61-5e95-4dc0-9172-2b49268fcbbd admin admin] REQUEST_METHOD: `PUT` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:04.675187 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bbeb61-5e95-4dc0-9172-2b49268fcbbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:04.675590 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bbeb61-5e95-4dc0-9172-2b49268fcbbd admin admin] PATH_INFO: `/v3/projects/17117d29c70040e28166c64f1fb10541/users/81fbfb70049343fea18013bf7c61c052/roles/02476a5b82eb4b188c18a5d706a3ef68` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:04.677567 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 929/1857] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:41:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:41:04.713596 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bbeb61-5e95-4dc0-9172-2b49268fcbbd admin admin] RBAC: Authorizing `identity:create_grant(project_id=17117d29c70040e28166c64f1fb10541, user_id=81fbfb70049343fea18013bf7c61c052, role_id=02476a5b82eb4b188c18a5d706a3ef68)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:04.717671 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bbeb61-5e95-4dc0-9172-2b49268fcbbd admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:04.736171 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 929/1858] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 17:41:04 2022] PUT /identity/v3/projects/17117d29c70040e28166c64f1fb10541/users/81fbfb70049343fea18013bf7c61c052/roles/02476a5b82eb4b188c18a5d706a3ef68 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:41:04.747047 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c31f291-f229-44a3-864e-a9d62c4e8ca7 None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:04.747338 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c31f291-f229-44a3-864e-a9d62c4e8ca7 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:04.747620 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c31f291-f229-44a3-864e-a9d62c4e8ca7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:04.782350 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bbeb61-5e95-4dc0-9172-2b49268fcbbd admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:04.785186 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-06bbeb61-5e95-4dc0-9172-2b49268fcbbd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:04.789625 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-3c31f291-f229-44a3-864e-a9d62c4e8ca7 None None] MFA Rules not processed for user `81fbfb70049343fea18013bf7c61c052`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:41:04.790843 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-3c31f291-f229-44a3-864e-a9d62c4e8ca7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:04.871221 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 930/1859] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:41:04 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:41:04.877327 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bbeb61-5e95-4dc0-9172-2b49268fcbbd admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:04.882467 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c31f291-f229-44a3-864e-a9d62c4e8ca7 None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:04.883194 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a36aa5-7ef3-4479-8494-b54229e258dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:04.883915 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a36aa5-7ef3-4479-8494-b54229e258dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:04.884030 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-3c31f291-f229-44a3-864e-a9d62c4e8ca7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:04.884329 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a36aa5-7ef3-4479-8494-b54229e258dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:04.884714 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a36aa5-7ef3-4479-8494-b54229e258dd admin admin] PATH_INFO: `/v3/users/bbf2966ef211410d9f3399015ce05b86` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:04.886042 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a36aa5-7ef3-4479-8494-b54229e258dd admin admin] RBAC: Authorizing `identity:delete_user(user_id=bbf2966ef211410d9f3399015ce05b86)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:04.888747 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a36aa5-7ef3-4479-8494-b54229e258dd admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:04.971848 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-01a36aa5-7ef3-4479-8494-b54229e258dd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bbf2966ef211410d9f3399015ce05b86'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:04.975702 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-01a36aa5-7ef3-4479-8494-b54229e258dd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bbf2966ef211410d9f3399015ce05b86'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:04.983734 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c31f291-f229-44a3-864e-a9d62c4e8ca7 None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:04.983944 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-01a36aa5-7ef3-4479-8494-b54229e258dd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bbf2966ef211410d9f3399015ce05b86'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:04.984472 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-01a36aa5-7ef3-4479-8494-b54229e258dd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bbf2966ef211410d9f3399015ce05b86'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:04.988809 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82bad3e-cb7c-4fea-b383-c9ac4f534d22 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59be342541264942ad87f60ff4de6c8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:04.990166 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82bad3e-cb7c-4fea-b383-c9ac4f534d22 service cinder] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:04.990800 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82bad3e-cb7c-4fea-b383-c9ac4f534d22 service cinder] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:04.991256 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82bad3e-cb7c-4fea-b383-c9ac4f534d22 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:04.993607 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-d82bad3e-cb7c-4fea-b383-c9ac4f534d22 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:04.994618 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-01a36aa5-7ef3-4479-8494-b54229e258dd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bbf2966ef211410d9f3399015ce05b86'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:05.011525 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82bad3e-cb7c-4fea-b383-c9ac4f534d22 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:05.013486 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82bad3e-cb7c-4fea-b383-c9ac4f534d22 service cinder] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:05.016662 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 930/1860] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 17:41:04 2022] DELETE /identity/v3/users/bbf2966ef211410d9f3399015ce05b86 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:41:05.022913 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178297f9-6a97-4b73-946f-82fef850b1ab None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:05.023498 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178297f9-6a97-4b73-946f-82fef850b1ab None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:05.023975 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178297f9-6a97-4b73-946f-82fef850b1ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:05.026361 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 931/1861] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:41:04 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:41:05.046355 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-178297f9-6a97-4b73-946f-82fef850b1ab None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:41:05.047781 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-178297f9-6a97-4b73-946f-82fef850b1ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:05.086237 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:41:05.129255 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 931/1862] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:41:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:41:05.186471 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82bad3e-cb7c-4fea-b383-c9ac4f534d22 service cinder] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:05.190443 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-d82bad3e-cb7c-4fea-b383-c9ac4f534d22 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:05.244243 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178297f9-6a97-4b73-946f-82fef850b1ab None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:05.253564 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-178297f9-6a97-4b73-946f-82fef850b1ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:05.295859 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178297f9-6a97-4b73-946f-82fef850b1ab None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:05.326035 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64eecfad-20aa-453e-84f1-961c3cf0f8cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:05.326741 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64eecfad-20aa-453e-84f1-961c3cf0f8cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:05.327156 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64eecfad-20aa-453e-84f1-961c3cf0f8cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:05.328135 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64eecfad-20aa-453e-84f1-961c3cf0f8cd admin admin] PATH_INFO: `/v3/projects/54ebee69ad68418d8a87fe5ea7a73547` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:05.330672 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64eecfad-20aa-453e-84f1-961c3cf0f8cd admin admin] RBAC: Authorizing `identity:delete_project(project_id=54ebee69ad68418d8a87fe5ea7a73547)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:05.340583 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64eecfad-20aa-453e-84f1-961c3cf0f8cd admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:05.427044 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82bad3e-cb7c-4fea-b383-c9ac4f534d22 service cinder] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:05.493571 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:41:05.508941 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-64eecfad-20aa-453e-84f1-961c3cf0f8cd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '54ebee69ad68418d8a87fe5ea7a73547', 'request_id': 'req-64eecfad-20aa-453e-84f1-961c3cf0f8cd'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:05.523063 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ad0f94-06b0-4cc5-90b9-2b92e5729426 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43ad98a250a144099ccd18378aedd4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:05.524248 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ad0f94-06b0-4cc5-90b9-2b92e5729426 service neutron] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:05.524713 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ad0f94-06b0-4cc5-90b9-2b92e5729426 service neutron] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:05.525262 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ad0f94-06b0-4cc5-90b9-2b92e5729426 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:05.531382 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-64eecfad-20aa-453e-84f1-961c3cf0f8cd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '54ebee69ad68418d8a87fe5ea7a73547', 'request_id': 'req-64eecfad-20aa-453e-84f1-961c3cf0f8cd'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:05.542183 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ad0f94-06b0-4cc5-90b9-2b92e5729426 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:05.544256 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ad0f94-06b0-4cc5-90b9-2b92e5729426 service neutron] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:05.549522 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-64eecfad-20aa-453e-84f1-961c3cf0f8cd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '54ebee69ad68418d8a87fe5ea7a73547', 'request_id': 'req-64eecfad-20aa-453e-84f1-961c3cf0f8cd'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:05.556828 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 932/1863] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:41:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 17:41:05.575530 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-64eecfad-20aa-453e-84f1-961c3cf0f8cd admin admin] The token cache is being invalidate because project 54ebee69ad68418d8a87fe5ea7a73547 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81759) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:41:05.581618 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-64eecfad-20aa-453e-84f1-961c3cf0f8cd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:05.582958 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-64eecfad-20aa-453e-84f1-961c3cf0f8cd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:05.584942 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 932/1864] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 17:41:05 2022] DELETE /identity/v3/projects/54ebee69ad68418d8a87fe5ea7a73547 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:41:05.588894 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ad0f94-06b0-4cc5-90b9-2b92e5729426 service neutron] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:05.607721 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ad0f94-06b0-4cc5-90b9-2b92e5729426 service neutron] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:05.685803 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a18abc4-9e15-4601-a019-4406e31398ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:05.686526 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a18abc4-9e15-4601-a019-4406e31398ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:05.686873 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a18abc4-9e15-4601-a019-4406e31398ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:05.687216 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a18abc4-9e15-4601-a019-4406e31398ce admin admin] PATH_INFO: `/v3/users/456e9f30853c40eb864e8cbd1f05cb65` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:05.692981 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a18abc4-9e15-4601-a019-4406e31398ce admin admin] RBAC: Authorizing `identity:delete_user(user_id=456e9f30853c40eb864e8cbd1f05cb65)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:05.696642 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a18abc4-9e15-4601-a019-4406e31398ce admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:05.828877 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-9a18abc4-9e15-4601-a019-4406e31398ce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '456e9f30853c40eb864e8cbd1f05cb65'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:05.829616 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-9a18abc4-9e15-4601-a019-4406e31398ce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '456e9f30853c40eb864e8cbd1f05cb65'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:05.839067 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-9a18abc4-9e15-4601-a019-4406e31398ce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '456e9f30853c40eb864e8cbd1f05cb65'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:05.840408 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-9a18abc4-9e15-4601-a019-4406e31398ce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '456e9f30853c40eb864e8cbd1f05cb65'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:05.860462 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-9a18abc4-9e15-4601-a019-4406e31398ce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '456e9f30853c40eb864e8cbd1f05cb65'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:05.882772 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 933/1865] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 17:41:05 2022] DELETE /identity/v3/users/456e9f30853c40eb864e8cbd1f05cb65 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:41:06.601948 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64eecfad-20aa-453e-84f1-961c3cf0f8cd admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:06.603419 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a18abc4-9e15-4601-a019-4406e31398ce admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:06.604820 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-64eecfad-20aa-453e-84f1-961c3cf0f8cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:06.605156 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-9a18abc4-9e15-4601-a019-4406e31398ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:06.621280 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64eecfad-20aa-453e-84f1-961c3cf0f8cd admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:06.685939 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:41:06.690622 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a18abc4-9e15-4601-a019-4406e31398ce admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:06.697479 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2daee5c9-d0c2-4ab1-b675-05c7eda8f9e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:06.698263 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2daee5c9-d0c2-4ab1-b675-05c7eda8f9e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:06.698593 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2daee5c9-d0c2-4ab1-b675-05c7eda8f9e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:06.698876 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2daee5c9-d0c2-4ab1-b675-05c7eda8f9e9 admin admin] PATH_INFO: `/v3/projects/59f0a6adaaf244a2965df3b0d6e5e95d` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:06.709122 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2daee5c9-d0c2-4ab1-b675-05c7eda8f9e9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=59f0a6adaaf244a2965df3b0d6e5e95d)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:06.711170 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2daee5c9-d0c2-4ab1-b675-05c7eda8f9e9 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:06.739183 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbd1b09-87bd-43a1-b4c7-b5a816122f6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:06.739923 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbd1b09-87bd-43a1-b4c7-b5a816122f6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:06.740738 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbd1b09-87bd-43a1-b4c7-b5a816122f6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:06.741727 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbd1b09-87bd-43a1-b4c7-b5a816122f6b admin admin] PATH_INFO: `/v3/projects/533620c1a10743248fdf05bc5d8b6002` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:06.744314 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbd1b09-87bd-43a1-b4c7-b5a816122f6b admin admin] RBAC: Authorizing `identity:delete_project(project_id=533620c1a10743248fdf05bc5d8b6002)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:06.748014 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbd1b09-87bd-43a1-b4c7-b5a816122f6b admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:06.783307 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-2daee5c9-d0c2-4ab1-b675-05c7eda8f9e9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '59f0a6adaaf244a2965df3b0d6e5e95d', 'request_id': 'req-2daee5c9-d0c2-4ab1-b675-05c7eda8f9e9'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:06.816843 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-2daee5c9-d0c2-4ab1-b675-05c7eda8f9e9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '59f0a6adaaf244a2965df3b0d6e5e95d', 'request_id': 'req-2daee5c9-d0c2-4ab1-b675-05c7eda8f9e9'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:06.833716 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-2daee5c9-d0c2-4ab1-b675-05c7eda8f9e9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '59f0a6adaaf244a2965df3b0d6e5e95d', 'request_id': 'req-2daee5c9-d0c2-4ab1-b675-05c7eda8f9e9'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:06.859077 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-2daee5c9-d0c2-4ab1-b675-05c7eda8f9e9 admin admin] The token cache is being invalidate because project 59f0a6adaaf244a2965df3b0d6e5e95d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81758) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:41:06.859800 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-2daee5c9-d0c2-4ab1-b675-05c7eda8f9e9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:06.862623 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-2daee5c9-d0c2-4ab1-b675-05c7eda8f9e9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:06.867559 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 934/1866] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 17:41:06 2022] DELETE /identity/v3/projects/59f0a6adaaf244a2965df3b0d6e5e95d => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:41:06.868247 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2daee5c9-d0c2-4ab1-b675-05c7eda8f9e9 admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:06.871136 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-2daee5c9-d0c2-4ab1-b675-05c7eda8f9e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:06.891479 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-8fbd1b09-87bd-43a1-b4c7-b5a816122f6b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '533620c1a10743248fdf05bc5d8b6002', 'request_id': 'req-8fbd1b09-87bd-43a1-b4c7-b5a816122f6b'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:06.900052 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-8fbd1b09-87bd-43a1-b4c7-b5a816122f6b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '533620c1a10743248fdf05bc5d8b6002', 'request_id': 'req-8fbd1b09-87bd-43a1-b4c7-b5a816122f6b'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:06.908825 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-8fbd1b09-87bd-43a1-b4c7-b5a816122f6b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '533620c1a10743248fdf05bc5d8b6002', 'request_id': 'req-8fbd1b09-87bd-43a1-b4c7-b5a816122f6b'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:06.910621 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:41:06.927685 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-8fbd1b09-87bd-43a1-b4c7-b5a816122f6b admin admin] The token cache is being invalidate because project 533620c1a10743248fdf05bc5d8b6002 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81759) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:41:06.927984 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-8fbd1b09-87bd-43a1-b4c7-b5a816122f6b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:06.928789 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-8fbd1b09-87bd-43a1-b4c7-b5a816122f6b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:06.933593 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 933/1867] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 17:41:06 2022] DELETE /identity/v3/projects/533620c1a10743248fdf05bc5d8b6002 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:41:06.933593 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbd1b09-87bd-43a1-b4c7-b5a816122f6b admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:06.933593 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-8fbd1b09-87bd-43a1-b4c7-b5a816122f6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:06.965252 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2daee5c9-d0c2-4ab1-b675-05c7eda8f9e9 admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:07.009473 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbd1b09-87bd-43a1-b4c7-b5a816122f6b admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:07.016044 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72874c1-9875-42ad-b03b-7301209b5ae9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:07.017693 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72874c1-9875-42ad-b03b-7301209b5ae9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:07.017693 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72874c1-9875-42ad-b03b-7301209b5ae9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:07.018112 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72874c1-9875-42ad-b03b-7301209b5ae9 admin admin] PATH_INFO: `/v3/users/2311da6de24e48999351c274db85e714` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:07.019409 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72874c1-9875-42ad-b03b-7301209b5ae9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2311da6de24e48999351c274db85e714)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:07.021707 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72874c1-9875-42ad-b03b-7301209b5ae9 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:07.027557 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e65f09-78d7-4cb5-aac0-d9fb15a4786d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:07.028201 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e65f09-78d7-4cb5-aac0-d9fb15a4786d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:07.028513 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e65f09-78d7-4cb5-aac0-d9fb15a4786d admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:07.028813 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e65f09-78d7-4cb5-aac0-d9fb15a4786d admin admin] PATH_INFO: `/v3/users/deaca783962e4cedb863f5bba76a4344` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:07.029951 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e65f09-78d7-4cb5-aac0-d9fb15a4786d admin admin] RBAC: Authorizing `identity:delete_user(user_id=deaca783962e4cedb863f5bba76a4344)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:07.031866 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e65f09-78d7-4cb5-aac0-d9fb15a4786d admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:07.069258 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-e72874c1-9875-42ad-b03b-7301209b5ae9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2311da6de24e48999351c274db85e714'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:07.069849 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-e72874c1-9875-42ad-b03b-7301209b5ae9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2311da6de24e48999351c274db85e714'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:07.071094 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:41:07.077569 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-e72874c1-9875-42ad-b03b-7301209b5ae9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2311da6de24e48999351c274db85e714'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:07.078238 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-e72874c1-9875-42ad-b03b-7301209b5ae9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2311da6de24e48999351c274db85e714'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:07.082577 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-e3e65f09-78d7-4cb5-aac0-d9fb15a4786d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'deaca783962e4cedb863f5bba76a4344'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:07.083477 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-e3e65f09-78d7-4cb5-aac0-d9fb15a4786d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'deaca783962e4cedb863f5bba76a4344'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:07.085927 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-e72874c1-9875-42ad-b03b-7301209b5ae9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2311da6de24e48999351c274db85e714'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:07.091962 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-e3e65f09-78d7-4cb5-aac0-d9fb15a4786d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'deaca783962e4cedb863f5bba76a4344'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:07.092977 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-e3e65f09-78d7-4cb5-aac0-d9fb15a4786d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'deaca783962e4cedb863f5bba76a4344'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:07.105219 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 934/1868] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 17:41:06 2022] DELETE /identity/v3/users/2311da6de24e48999351c274db85e714 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:41:07.108437 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b8174e-33c9-47fa-ba1c-b44503fa69df None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:07.108437 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b8174e-33c9-47fa-ba1c-b44503fa69df None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:07.108437 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b8174e-33c9-47fa-ba1c-b44503fa69df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:07.113182 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-e3e65f09-78d7-4cb5-aac0-d9fb15a4786d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'deaca783962e4cedb863f5bba76a4344'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:07.126766 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-67b8174e-33c9-47fa-ba1c-b44503fa69df None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:41:07.128193 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-67b8174e-33c9-47fa-ba1c-b44503fa69df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:07.129841 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 935/1869] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 17:41:06 2022] DELETE /identity/v3/users/deaca783962e4cedb863f5bba76a4344 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:41:07.134264 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42f86e3-c957-4fdc-b1de-86af6a62b789 None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:07.134567 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42f86e3-c957-4fdc-b1de-86af6a62b789 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:07.134922 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42f86e3-c957-4fdc-b1de-86af6a62b789 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:07.157134 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-f42f86e3-c957-4fdc-b1de-86af6a62b789 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:41:07.158952 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-f42f86e3-c957-4fdc-b1de-86af6a62b789 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:07.201544 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 935/1870] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:41:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:41:07.210965 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bfb55c-466c-41cc-a79a-4fb63ac454ff None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:07.211252 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bfb55c-466c-41cc-a79a-4fb63ac454ff None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:07.211529 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bfb55c-466c-41cc-a79a-4fb63ac454ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:07.235328 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-53bfb55c-466c-41cc-a79a-4fb63ac454ff None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:41:07.235499 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 936/1871] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:41:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:41:07.237262 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-53bfb55c-466c-41cc-a79a-4fb63ac454ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:07.245325 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42f86e3-c957-4fdc-b1de-86af6a62b789 None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:07.246669 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-f42f86e3-c957-4fdc-b1de-86af6a62b789 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:07.251792 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 936/1872] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:41:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:41:07.255736 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bfb55c-466c-41cc-a79a-4fb63ac454ff None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:07.261978 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:INSERT=2,keystone:DELETE=11,keystone:SELECT=87 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:41:07.266266 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bfb55c-466c-41cc-a79a-4fb63ac454ff None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:07.280342 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc051ab6-8a65-4325-9ba5-de1cf582dc8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:07.281682 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc051ab6-8a65-4325-9ba5-de1cf582dc8b admin admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:07.281682 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc051ab6-8a65-4325-9ba5-de1cf582dc8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:07.281682 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc051ab6-8a65-4325-9ba5-de1cf582dc8b admin admin] PATH_INFO: `/v3/domains` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:07.282283 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc051ab6-8a65-4325-9ba5-de1cf582dc8b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:41:07.282583 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc051ab6-8a65-4325-9ba5-de1cf582dc8b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:07.285596 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc051ab6-8a65-4325-9ba5-de1cf582dc8b admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:07.301287 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 937/1873] 10.222.0.45 () {58 vars in 1153 bytes} [Wed Aug 24 17:41:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:41:07.305810 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc051ab6-8a65-4325-9ba5-de1cf582dc8b admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:07.317074 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc051ab6-8a65-4325-9ba5-de1cf582dc8b admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:07.322213 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c362aa7-0beb-412a-9668-8a9b175d8a9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:07.322832 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c362aa7-0beb-412a-9668-8a9b175d8a9c admin admin] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:07.323530 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c362aa7-0beb-412a-9668-8a9b175d8a9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:07.323530 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c362aa7-0beb-412a-9668-8a9b175d8a9c admin admin] PATH_INFO: `/v3/projects` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:07.324030 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42f86e3-c957-4fdc-b1de-86af6a62b789 None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:07.324142 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c362aa7-0beb-412a-9668-8a9b175d8a9c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:07.326323 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c362aa7-0beb-412a-9668-8a9b175d8a9c admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:07.327996 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2141735-2e17-4611-ab3e-955cbbd8a0bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43ad98a250a144099ccd18378aedd4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:07.328835 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2141735-2e17-4611-ab3e-955cbbd8a0bc service neutron] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:07.329303 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2141735-2e17-4611-ab3e-955cbbd8a0bc service neutron] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:07.330447 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2141735-2e17-4611-ab3e-955cbbd8a0bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:07.342091 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2141735-2e17-4611-ab3e-955cbbd8a0bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:07.343755 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2141735-2e17-4611-ab3e-955cbbd8a0bc service neutron] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:07.365044 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 938/1874] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 17:41:07 2022] POST /identity/v3/projects => generated 409 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:41:07.367402 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c362aa7-0beb-412a-9668-8a9b175d8a9c admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:07.378052 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 937/1875] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:41:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:41:07.436493 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c362aa7-0beb-412a-9668-8a9b175d8a9c admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:07.443055 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bec5030-a4e2-4c38-a368-30b4fe9949df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:07.443716 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bec5030-a4e2-4c38-a368-30b4fe9949df admin admin] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:07.444075 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bec5030-a4e2-4c38-a368-30b4fe9949df admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:07.444406 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bec5030-a4e2-4c38-a368-30b4fe9949df admin admin] PATH_INFO: `/v3/users` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:07.445077 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bec5030-a4e2-4c38-a368-30b4fe9949df admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:07.447034 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bec5030-a4e2-4c38-a368-30b4fe9949df admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:07.470255 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 939/1876] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 17:41:07 2022] POST /identity/v3/users => generated 305 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:41:07.474681 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2141735-2e17-4611-ab3e-955cbbd8a0bc service neutron] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:07.486893 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2141735-2e17-4611-ab3e-955cbbd8a0bc service neutron] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:07.492526 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fb8d82-1336-43b0-a435-831f53876840 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:07.493165 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fb8d82-1336-43b0-a435-831f53876840 admin admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:07.493450 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fb8d82-1336-43b0-a435-831f53876840 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:07.493726 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fb8d82-1336-43b0-a435-831f53876840 admin admin] PATH_INFO: `/v3/roles` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:07.494372 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fb8d82-1336-43b0-a435-831f53876840 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:07.496010 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fb8d82-1336-43b0-a435-831f53876840 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:07.510514 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 938/1877] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:41:07 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 17:41:07.511310 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fb8d82-1336-43b0-a435-831f53876840 admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:07.512656 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bec5030-a4e2-4c38-a368-30b4fe9949df admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:07.514715 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-5bec5030-a4e2-4c38-a368-30b4fe9949df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:07.523724 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fb8d82-1336-43b0-a435-831f53876840 admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:07.528747 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59e1ff2-bcf4-4922-91c4-d3e3a50d9713 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:07.529592 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59e1ff2-bcf4-4922-91c4-d3e3a50d9713 admin admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:07.529913 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59e1ff2-bcf4-4922-91c4-d3e3a50d9713 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:07.530260 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59e1ff2-bcf4-4922-91c4-d3e3a50d9713 admin admin] PATH_INFO: `/v3/roles` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:07.531263 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59e1ff2-bcf4-4922-91c4-d3e3a50d9713 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:07.533369 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bec5030-a4e2-4c38-a368-30b4fe9949df admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:07.533708 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59e1ff2-bcf4-4922-91c4-d3e3a50d9713 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:07.539643 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220bb7c9-28fa-4fb5-a8b4-31c33fecb312 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:07.540373 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220bb7c9-28fa-4fb5-a8b4-31c33fecb312 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:07.540715 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220bb7c9-28fa-4fb5-a8b4-31c33fecb312 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:07.541014 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220bb7c9-28fa-4fb5-a8b4-31c33fecb312 admin admin] PATH_INFO: `/v3/projects/dc7407cc1aee45f5bb60119df71fb64c` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:07.542527 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220bb7c9-28fa-4fb5-a8b4-31c33fecb312 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dc7407cc1aee45f5bb60119df71fb64c)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:07.544459 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220bb7c9-28fa-4fb5-a8b4-31c33fecb312 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:07.545106 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 939/1878] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:41:07 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:41:07.549024 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59e1ff2-bcf4-4922-91c4-d3e3a50d9713 admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:07.577557 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59e1ff2-bcf4-4922-91c4-d3e3a50d9713 admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:07.586247 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7d8f03-5adf-4c12-9b48-c57650c81d92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:07.587066 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7d8f03-5adf-4c12-9b48-c57650c81d92 admin admin] REQUEST_METHOD: `PUT` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:07.587370 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7d8f03-5adf-4c12-9b48-c57650c81d92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:07.587683 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7d8f03-5adf-4c12-9b48-c57650c81d92 admin admin] PATH_INFO: `/v3/projects/b035c069f2624ff2851fee7414f8c390/users/cd19c2803225410988976133b98bcf30/roles/02476a5b82eb4b188c18a5d706a3ef68` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:07.606638 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7d8f03-5adf-4c12-9b48-c57650c81d92 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b035c069f2624ff2851fee7414f8c390, user_id=cd19c2803225410988976133b98bcf30, role_id=02476a5b82eb4b188c18a5d706a3ef68)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:07.609381 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7d8f03-5adf-4c12-9b48-c57650c81d92 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:07.615824 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-220bb7c9-28fa-4fb5-a8b4-31c33fecb312 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc7407cc1aee45f5bb60119df71fb64c', 'request_id': 'req-220bb7c9-28fa-4fb5-a8b4-31c33fecb312'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:07.622299 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-220bb7c9-28fa-4fb5-a8b4-31c33fecb312 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc7407cc1aee45f5bb60119df71fb64c', 'request_id': 'req-220bb7c9-28fa-4fb5-a8b4-31c33fecb312'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:07.625728 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 940/1879] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 17:41:07 2022] PUT /identity/v3/projects/b035c069f2624ff2851fee7414f8c390/users/cd19c2803225410988976133b98bcf30/roles/02476a5b82eb4b188c18a5d706a3ef68 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:41:07.630152 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-220bb7c9-28fa-4fb5-a8b4-31c33fecb312 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc7407cc1aee45f5bb60119df71fb64c', 'request_id': 'req-220bb7c9-28fa-4fb5-a8b4-31c33fecb312'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:07.636952 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91029984-4210-421f-9bc6-ab5a64d4e179 None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:07.637910 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91029984-4210-421f-9bc6-ab5a64d4e179 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:07.637910 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91029984-4210-421f-9bc6-ab5a64d4e179 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:07.644467 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-220bb7c9-28fa-4fb5-a8b4-31c33fecb312 admin admin] The token cache is being invalidate because project dc7407cc1aee45f5bb60119df71fb64c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81759) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:41:07.644916 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-220bb7c9-28fa-4fb5-a8b4-31c33fecb312 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:07.647212 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-220bb7c9-28fa-4fb5-a8b4-31c33fecb312 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:07.649265 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 940/1880] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 17:41:07 2022] DELETE /identity/v3/projects/dc7407cc1aee45f5bb60119df71fb64c => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:41:07.653950 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78e0d04-8e03-4e75-ba25-4a28d36b3e68 None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:07.654935 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78e0d04-8e03-4e75-ba25-4a28d36b3e68 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:07.655668 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78e0d04-8e03-4e75-ba25-4a28d36b3e68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:07.664912 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:41:07.684959 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-e78e0d04-8e03-4e75-ba25-4a28d36b3e68 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:41:07.685520 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-91029984-4210-421f-9bc6-ab5a64d4e179 None None] MFA Rules not processed for user `cd19c2803225410988976133b98bcf30`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:41:07.687283 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-e78e0d04-8e03-4e75-ba25-4a28d36b3e68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:07.687495 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-91029984-4210-421f-9bc6-ab5a64d4e179 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:07.735240 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:41:07.767170 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 941/1881] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:41:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:41:07.775125 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81288ec4-5b13-44f9-a4dd-22ec048ccf32 None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:07.775409 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81288ec4-5b13-44f9-a4dd-22ec048ccf32 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:07.775685 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81288ec4-5b13-44f9-a4dd-22ec048ccf32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:07.812646 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-81288ec4-5b13-44f9-a4dd-22ec048ccf32 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:41:07.813889 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-81288ec4-5b13-44f9-a4dd-22ec048ccf32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:07.824477 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 942/1882] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:41:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:41:07.828705 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81288ec4-5b13-44f9-a4dd-22ec048ccf32 None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:07.831218 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 941/1883] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:41:07 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:41:07.835414 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91029984-4210-421f-9bc6-ab5a64d4e179 None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:07.836878 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-91029984-4210-421f-9bc6-ab5a64d4e179 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:07.849913 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81288ec4-5b13-44f9-a4dd-22ec048ccf32 None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:07.853935 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91029984-4210-421f-9bc6-ab5a64d4e179 None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:07.856323 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491daa96-9ada-497c-932f-1c85cc5ab695 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:07.856983 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491daa96-9ada-497c-932f-1c85cc5ab695 admin admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:07.857259 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491daa96-9ada-497c-932f-1c85cc5ab695 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:07.857640 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491daa96-9ada-497c-932f-1c85cc5ab695 admin admin] PATH_INFO: `/v3/domains` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:07.858393 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491daa96-9ada-497c-932f-1c85cc5ab695 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:41:07.858910 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491daa96-9ada-497c-932f-1c85cc5ab695 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:07.859104 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab46a58-415d-46be-8b4a-78ece59f2455 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:07.859746 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab46a58-415d-46be-8b4a-78ece59f2455 admin admin] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:07.860036 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab46a58-415d-46be-8b4a-78ece59f2455 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:07.860312 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab46a58-415d-46be-8b4a-78ece59f2455 admin admin] PATH_INFO: `/v3/projects` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:07.860693 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491daa96-9ada-497c-932f-1c85cc5ab695 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:07.861039 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab46a58-415d-46be-8b4a-78ece59f2455 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:07.863184 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab46a58-415d-46be-8b4a-78ece59f2455 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:07.875744 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 943/1884] 10.222.0.45 () {58 vars in 1153 bytes} [Wed Aug 24 17:41:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:41:07.880026 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491daa96-9ada-497c-932f-1c85cc5ab695 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:07.893093 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491daa96-9ada-497c-932f-1c85cc5ab695 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:07.899199 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9cc0e4-4801-4ba1-ae95-250deec6434c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:07.900238 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9cc0e4-4801-4ba1-ae95-250deec6434c admin admin] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:07.900796 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9cc0e4-4801-4ba1-ae95-250deec6434c admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:07.902138 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9cc0e4-4801-4ba1-ae95-250deec6434c admin admin] PATH_INFO: `/v3/projects` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:07.902970 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9cc0e4-4801-4ba1-ae95-250deec6434c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:07.905103 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9cc0e4-4801-4ba1-ae95-250deec6434c admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:07.927107 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 942/1885] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 17:41:07 2022] POST /identity/v3/projects => generated 407 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:41:07.932585 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab46a58-415d-46be-8b4a-78ece59f2455 admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:07.946712 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 944/1886] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 17:41:07 2022] POST /identity/v3/projects => generated 387 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:41:07.947638 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9cc0e4-4801-4ba1-ae95-250deec6434c admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:07.949521 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-eb9cc0e4-4801-4ba1-ae95-250deec6434c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:08.032198 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab46a58-415d-46be-8b4a-78ece59f2455 admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:08.037697 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f04a2e-9cac-449b-9e91-52c4f98b1c75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:08.038321 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f04a2e-9cac-449b-9e91-52c4f98b1c75 admin admin] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:08.038605 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f04a2e-9cac-449b-9e91-52c4f98b1c75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:08.038887 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f04a2e-9cac-449b-9e91-52c4f98b1c75 admin admin] PATH_INFO: `/v3/users` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:08.039525 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f04a2e-9cac-449b-9e91-52c4f98b1c75 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:08.039875 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9cc0e4-4801-4ba1-ae95-250deec6434c admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:08.041494 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f04a2e-9cac-449b-9e91-52c4f98b1c75 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:08.051038 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438efa15-7f06-4fbd-a4ba-623a944251b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:08.051794 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438efa15-7f06-4fbd-a4ba-623a944251b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:08.052238 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438efa15-7f06-4fbd-a4ba-623a944251b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:08.052517 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438efa15-7f06-4fbd-a4ba-623a944251b4 admin admin] PATH_INFO: `/v3/users/81fbfb70049343fea18013bf7c61c052` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:08.054212 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438efa15-7f06-4fbd-a4ba-623a944251b4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=81fbfb70049343fea18013bf7c61c052)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:08.056493 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438efa15-7f06-4fbd-a4ba-623a944251b4 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:08.070872 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 943/1887] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 17:41:07 2022] POST /identity/v3/users => generated 310 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:41:08.071802 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f04a2e-9cac-449b-9e91-52c4f98b1c75 admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:08.082973 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f04a2e-9cac-449b-9e91-52c4f98b1c75 admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:08.087831 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247eab08-0bee-429b-bb28-87b1a6b087eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:08.088463 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247eab08-0bee-429b-bb28-87b1a6b087eb admin admin] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:08.088778 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247eab08-0bee-429b-bb28-87b1a6b087eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:08.089060 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247eab08-0bee-429b-bb28-87b1a6b087eb admin admin] PATH_INFO: `/v3/users` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:08.089707 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247eab08-0bee-429b-bb28-87b1a6b087eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:08.091664 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247eab08-0bee-429b-bb28-87b1a6b087eb admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:08.121565 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 944/1888] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 17:41:08 2022] POST /identity/v3/users => generated 294 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:41:08.123745 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247eab08-0bee-429b-bb28-87b1a6b087eb admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:08.130665 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-247eab08-0bee-429b-bb28-87b1a6b087eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:08.143159 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-438efa15-7f06-4fbd-a4ba-623a944251b4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '81fbfb70049343fea18013bf7c61c052'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:08.144229 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-438efa15-7f06-4fbd-a4ba-623a944251b4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '81fbfb70049343fea18013bf7c61c052'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:08.149485 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247eab08-0bee-429b-bb28-87b1a6b087eb admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:08.154848 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2760cee2-fb72-4688-87e3-919657a5d3f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:08.155529 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2760cee2-fb72-4688-87e3-919657a5d3f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:08.155817 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2760cee2-fb72-4688-87e3-919657a5d3f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:08.156098 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2760cee2-fb72-4688-87e3-919657a5d3f4 admin admin] PATH_INFO: `/v3/projects/13887aec17eb4598a8535dd7f5599d39` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:08.157293 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2760cee2-fb72-4688-87e3-919657a5d3f4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=13887aec17eb4598a8535dd7f5599d39)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:08.158799 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-438efa15-7f06-4fbd-a4ba-623a944251b4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '81fbfb70049343fea18013bf7c61c052'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:08.159425 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2760cee2-fb72-4688-87e3-919657a5d3f4 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:08.159692 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-438efa15-7f06-4fbd-a4ba-623a944251b4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '81fbfb70049343fea18013bf7c61c052'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:08.170598 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-438efa15-7f06-4fbd-a4ba-623a944251b4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '81fbfb70049343fea18013bf7c61c052'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:08.199843 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 945/1889] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 17:41:07 2022] DELETE /identity/v3/users/81fbfb70049343fea18013bf7c61c052 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:41:08.200462 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438efa15-7f06-4fbd-a4ba-623a944251b4 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:08.203471 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-438efa15-7f06-4fbd-a4ba-623a944251b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:08.230906 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=90,keystone:INSERT=7,keystone:DELETE=3 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:41:08.232865 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-2760cee2-fb72-4688-87e3-919657a5d3f4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '13887aec17eb4598a8535dd7f5599d39', 'request_id': 'req-2760cee2-fb72-4688-87e3-919657a5d3f4'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:08.239220 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:41:08.247191 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-2760cee2-fb72-4688-87e3-919657a5d3f4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '13887aec17eb4598a8535dd7f5599d39', 'request_id': 'req-2760cee2-fb72-4688-87e3-919657a5d3f4'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:08.258210 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-2760cee2-fb72-4688-87e3-919657a5d3f4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '13887aec17eb4598a8535dd7f5599d39', 'request_id': 'req-2760cee2-fb72-4688-87e3-919657a5d3f4'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:08.274485 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-2760cee2-fb72-4688-87e3-919657a5d3f4 admin admin] The token cache is being invalidate because project 13887aec17eb4598a8535dd7f5599d39 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81758) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:41:08.274784 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-2760cee2-fb72-4688-87e3-919657a5d3f4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:08.275464 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-2760cee2-fb72-4688-87e3-919657a5d3f4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:08.277058 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 945/1890] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 17:41:08 2022] DELETE /identity/v3/projects/13887aec17eb4598a8535dd7f5599d39 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:41:08.277962 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2760cee2-fb72-4688-87e3-919657a5d3f4 admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:08.279300 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-2760cee2-fb72-4688-87e3-919657a5d3f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:08.296543 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438efa15-7f06-4fbd-a4ba-623a944251b4 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:08.348225 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2760cee2-fb72-4688-87e3-919657a5d3f4 admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:08.352963 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8221199-6176-4d23-bafe-084e1232150c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:08.353724 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8221199-6176-4d23-bafe-084e1232150c admin admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:08.353921 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8221199-6176-4d23-bafe-084e1232150c admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:08.354194 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8221199-6176-4d23-bafe-084e1232150c admin admin] PATH_INFO: `/v3/roles` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:08.354840 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8221199-6176-4d23-bafe-084e1232150c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:08.356430 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8221199-6176-4d23-bafe-084e1232150c admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:08.358189 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e38a2dc-a441-4a5d-8677-a2a4e9f63630 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:08.358838 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e38a2dc-a441-4a5d-8677-a2a4e9f63630 admin admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:08.359119 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e38a2dc-a441-4a5d-8677-a2a4e9f63630 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:08.359395 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e38a2dc-a441-4a5d-8677-a2a4e9f63630 admin admin] PATH_INFO: `/v3/roles` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:08.360045 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e38a2dc-a441-4a5d-8677-a2a4e9f63630 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:08.361727 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e38a2dc-a441-4a5d-8677-a2a4e9f63630 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:08.366741 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 946/1891] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:41:08 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:41:08.368044 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38928e9a-90d0-41ca-983f-7276848f6fd3 None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:08.368334 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38928e9a-90d0-41ca-983f-7276848f6fd3 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:08.368633 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38928e9a-90d0-41ca-983f-7276848f6fd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:08.373492 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 946/1892] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:41:08 2022] GET /identity/v3/roles => generated 1420 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:41:08.374995 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a53333-75a5-442a-9e39-0e708ebf572d None None] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:08.375412 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a53333-75a5-442a-9e39-0e708ebf572d None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:08.375802 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a53333-75a5-442a-9e39-0e708ebf572d None None] PATH_INFO: `/` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:08.377613 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 947/1893] 10.222.0.45 () {54 vars in 921 bytes} [Wed Aug 24 17:41:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:41:08.378646 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a53333-75a5-442a-9e39-0e708ebf572d None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:08.380400 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-e6a53333-75a5-442a-9e39-0e708ebf572d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:08.390236 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-38928e9a-90d0-41ca-983f-7276848f6fd3 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:41:08.391441 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-38928e9a-90d0-41ca-983f-7276848f6fd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:08.397339 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a53333-75a5-442a-9e39-0e708ebf572d None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:08.399878 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 947/1894] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:41:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:41:08.400688 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38928e9a-90d0-41ca-983f-7276848f6fd3 None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:08.403091 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab2ba3d-576e-4613-bfc0-fc692f795202 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:08.404012 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab2ba3d-576e-4613-bfc0-fc692f795202 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:08.404012 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab2ba3d-576e-4613-bfc0-fc692f795202 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:08.404288 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab2ba3d-576e-4613-bfc0-fc692f795202 admin admin] PATH_INFO: `/v3/users/3f43ce59bbf94d189ca0ddc555e22bf3` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:08.405673 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab2ba3d-576e-4613-bfc0-fc692f795202 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3f43ce59bbf94d189ca0ddc555e22bf3)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:08.407560 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab2ba3d-576e-4613-bfc0-fc692f795202 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:08.410313 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38928e9a-90d0-41ca-983f-7276848f6fd3 None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:08.414955 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac97328-d5b1-47cc-baea-e52cb10b8914 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:08.415565 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac97328-d5b1-47cc-baea-e52cb10b8914 admin admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:08.415847 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac97328-d5b1-47cc-baea-e52cb10b8914 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:08.416122 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac97328-d5b1-47cc-baea-e52cb10b8914 admin admin] PATH_INFO: `/v3/roles` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:08.416790 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac97328-d5b1-47cc-baea-e52cb10b8914 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:08.418446 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac97328-d5b1-47cc-baea-e52cb10b8914 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:08.429170 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 948/1895] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:41:08 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:41:08.429960 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac97328-d5b1-47cc-baea-e52cb10b8914 admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:08.433223 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-dac97328-d5b1-47cc-baea-e52cb10b8914 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:08.448233 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac97328-d5b1-47cc-baea-e52cb10b8914 admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:08.453114 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ed93de-c46e-47ac-916a-b5264b6830fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:08.453808 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ed93de-c46e-47ac-916a-b5264b6830fd admin admin] REQUEST_METHOD: `PUT` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:08.454090 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ed93de-c46e-47ac-916a-b5264b6830fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:08.454368 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ed93de-c46e-47ac-916a-b5264b6830fd admin admin] PATH_INFO: `/v3/projects/42a6b11e6b0e4116bee451839753316f/users/7bf0ab87f39d4ae0aa21f87faff1ed6c/roles/e411ad9eb96e4e8d85b59b5b700ada7b` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:08.456996 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-5ab2ba3d-576e-4613-bfc0-fc692f795202 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3f43ce59bbf94d189ca0ddc555e22bf3'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:08.457744 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-5ab2ba3d-576e-4613-bfc0-fc692f795202 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3f43ce59bbf94d189ca0ddc555e22bf3'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:08.464782 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-5ab2ba3d-576e-4613-bfc0-fc692f795202 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3f43ce59bbf94d189ca0ddc555e22bf3'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:08.465440 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-5ab2ba3d-576e-4613-bfc0-fc692f795202 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3f43ce59bbf94d189ca0ddc555e22bf3'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:08.469643 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ed93de-c46e-47ac-916a-b5264b6830fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=42a6b11e6b0e4116bee451839753316f, user_id=7bf0ab87f39d4ae0aa21f87faff1ed6c, role_id=e411ad9eb96e4e8d85b59b5b700ada7b)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:08.473514 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ed93de-c46e-47ac-916a-b5264b6830fd admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:08.473979 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-5ab2ba3d-576e-4613-bfc0-fc692f795202 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3f43ce59bbf94d189ca0ddc555e22bf3'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:08.486708 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 949/1896] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 17:41:08 2022] PUT /identity/v3/projects/42a6b11e6b0e4116bee451839753316f/users/7bf0ab87f39d4ae0aa21f87faff1ed6c/roles/e411ad9eb96e4e8d85b59b5b700ada7b => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:41:08.488005 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78eaf8ce-2de4-46a8-9c3e-4062e7a9dffe None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:08.488395 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78eaf8ce-2de4-46a8-9c3e-4062e7a9dffe None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:08.488596 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78eaf8ce-2de4-46a8-9c3e-4062e7a9dffe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:08.494467 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 948/1897] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 17:41:08 2022] DELETE /identity/v3/users/3f43ce59bbf94d189ca0ddc555e22bf3 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:41:08.495553 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab2ba3d-576e-4613-bfc0-fc692f795202 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:08.498196 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-5ab2ba3d-576e-4613-bfc0-fc692f795202 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:08.508436 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-78eaf8ce-2de4-46a8-9c3e-4062e7a9dffe None None] MFA Rules not processed for user `053b7ce6d6f649ed91a26e434dab805e`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:41:08.509685 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-78eaf8ce-2de4-46a8-9c3e-4062e7a9dffe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:08.542777 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:41:08.564730 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab2ba3d-576e-4613-bfc0-fc692f795202 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:08.570775 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5b9507-2c88-4bdf-a6a6-11b7a08f6b76 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43ad98a250a144099ccd18378aedd4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:08.571398 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5b9507-2c88-4bdf-a6a6-11b7a08f6b76 service neutron] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:08.571674 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5b9507-2c88-4bdf-a6a6-11b7a08f6b76 service neutron] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:08.572262 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5b9507-2c88-4bdf-a6a6-11b7a08f6b76 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:08.574904 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-fe5b9507-2c88-4bdf-a6a6-11b7a08f6b76 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:08.586107 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 950/1898] 10.222.0.45 () {58 vars in 1018 bytes} [Wed Aug 24 17:41:08 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:41:08.587008 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78eaf8ce-2de4-46a8-9c3e-4062e7a9dffe None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:08.588148 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-78eaf8ce-2de4-46a8-9c3e-4062e7a9dffe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:08.615503 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:DELETE=4,keystone:SELECT=94,keystone:INSERT=2 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:41:08.651320 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5b9507-2c88-4bdf-a6a6-11b7a08f6b76 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:08.653540 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5b9507-2c88-4bdf-a6a6-11b7a08f6b76 service neutron] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:08.669319 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 949/1899] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:41:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:41:08.670222 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5b9507-2c88-4bdf-a6a6-11b7a08f6b76 service neutron] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:08.677511 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78eaf8ce-2de4-46a8-9c3e-4062e7a9dffe None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:08.681639 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-fe5b9507-2c88-4bdf-a6a6-11b7a08f6b76 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:08.683372 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbefbefd-2181-4983-99d8-c7a93f6944c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:08.684133 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbefbefd-2181-4983-99d8-c7a93f6944c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:08.684487 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbefbefd-2181-4983-99d8-c7a93f6944c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:08.684856 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbefbefd-2181-4983-99d8-c7a93f6944c2 admin admin] PATH_INFO: `/v3/projects/125bccf685c94f6db6cd666681fa1ff3/users/6873bc3749844100a2ad58c1160204bd/roles/02476a5b82eb4b188c18a5d706a3ef68` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:08.710038 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbefbefd-2181-4983-99d8-c7a93f6944c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=125bccf685c94f6db6cd666681fa1ff3, user_id=6873bc3749844100a2ad58c1160204bd, role_id=02476a5b82eb4b188c18a5d706a3ef68)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:08.714723 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbefbefd-2181-4983-99d8-c7a93f6944c2 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:08.715746 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5b9507-2c88-4bdf-a6a6-11b7a08f6b76 service neutron] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:08.730132 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4629d8b6-6350-44eb-aaab-6800e4b78405 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:08.731054 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4629d8b6-6350-44eb-aaab-6800e4b78405 admin admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:08.731336 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4629d8b6-6350-44eb-aaab-6800e4b78405 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:08.731678 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4629d8b6-6350-44eb-aaab-6800e4b78405 admin admin] PATH_INFO: `/v3/roles` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:08.732798 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4629d8b6-6350-44eb-aaab-6800e4b78405 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:08.734446 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4629d8b6-6350-44eb-aaab-6800e4b78405 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:08.739010 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 951/1900] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 17:41:08 2022] PUT /identity/v3/projects/125bccf685c94f6db6cd666681fa1ff3/users/6873bc3749844100a2ad58c1160204bd/roles/02476a5b82eb4b188c18a5d706a3ef68 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:41:08.740381 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbefbefd-2181-4983-99d8-c7a93f6944c2 admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:08.751292 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-bbefbefd-2181-4983-99d8-c7a93f6944c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:08.760864 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 950/1901] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:41:08 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:41:08.764781 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ce3dcc-474f-4b73-84a2-7eefd438b961 None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:08.765124 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ce3dcc-474f-4b73-84a2-7eefd438b961 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:08.765407 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ce3dcc-474f-4b73-84a2-7eefd438b961 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:08.819106 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-64ce3dcc-474f-4b73-84a2-7eefd438b961 None None] MFA Rules not processed for user `6873bc3749844100a2ad58c1160204bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:41:08.820390 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-64ce3dcc-474f-4b73-84a2-7eefd438b961 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:08.882321 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbefbefd-2181-4983-99d8-c7a93f6944c2 admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:08.888288 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069bd945-af69-4b0c-93ce-145cbaa0efa2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '053b7ce6d6f649ed91a26e434dab805e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:08.888960 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069bd945-af69-4b0c-93ce-145cbaa0efa2 service nova] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:08.889242 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069bd945-af69-4b0c-93ce-145cbaa0efa2 service nova] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:08.889518 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069bd945-af69-4b0c-93ce-145cbaa0efa2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:08.899572 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069bd945-af69-4b0c-93ce-145cbaa0efa2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:08.901519 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069bd945-af69-4b0c-93ce-145cbaa0efa2 service nova] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:08.916590 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 952/1902] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:41:08 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:41:08.916590 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069bd945-af69-4b0c-93ce-145cbaa0efa2 service nova] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:08.921834 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 951/1903] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:41:08 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:41:08.926730 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ce3dcc-474f-4b73-84a2-7eefd438b961 None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:09.023660 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ce3dcc-474f-4b73-84a2-7eefd438b961 None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:09.029425 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf69590-b85e-4bde-93b5-9b916468cf3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:09.030053 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf69590-b85e-4bde-93b5-9b916468cf3c admin admin] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:09.030336 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf69590-b85e-4bde-93b5-9b916468cf3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:09.030611 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf69590-b85e-4bde-93b5-9b916468cf3c admin admin] PATH_INFO: `/v3/projects` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:09.031250 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf69590-b85e-4bde-93b5-9b916468cf3c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:09.033822 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf69590-b85e-4bde-93b5-9b916468cf3c admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:09.059087 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:41:09.079708 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069bd945-af69-4b0c-93ce-145cbaa0efa2 service nova] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:09.085950 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b5bc41-8e36-47fc-85db-2a1076a2bf2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:09.088038 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b5bc41-8e36-47fc-85db-2a1076a2bf2b admin admin] REQUEST_METHOD: `PUT` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:09.089115 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b5bc41-8e36-47fc-85db-2a1076a2bf2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:09.090885 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b5bc41-8e36-47fc-85db-2a1076a2bf2b admin admin] PATH_INFO: `/v3/projects/42a6b11e6b0e4116bee451839753316f/users/7bf0ab87f39d4ae0aa21f87faff1ed6c/roles/e411ad9eb96e4e8d85b59b5b700ada7b` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:09.094381 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b5bc41-8e36-47fc-85db-2a1076a2bf2b admin admin] RBAC: Authorizing `identity:create_grant(project_id=42a6b11e6b0e4116bee451839753316f, user_id=7bf0ab87f39d4ae0aa21f87faff1ed6c, role_id=e411ad9eb96e4e8d85b59b5b700ada7b)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:09.097312 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 952/1904] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 17:41:08 2022] POST /identity/v3/projects => generated 385 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:41:09.098122 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b5bc41-8e36-47fc-85db-2a1076a2bf2b admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:09.101363 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf69590-b85e-4bde-93b5-9b916468cf3c admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:09.102908 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-1bf69590-b85e-4bde-93b5-9b916468cf3c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:09.117009 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 953/1905] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 17:41:08 2022] PUT /identity/v3/projects/42a6b11e6b0e4116bee451839753316f/users/7bf0ab87f39d4ae0aa21f87faff1ed6c/roles/e411ad9eb96e4e8d85b59b5b700ada7b => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:41:09.120870 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b5bc41-8e36-47fc-85db-2a1076a2bf2b admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:09.125177 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-b9b5bc41-8e36-47fc-85db-2a1076a2bf2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:09.162018 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:41:09.182246 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf69590-b85e-4bde-93b5-9b916468cf3c admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:09.213144 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b5bc41-8e36-47fc-85db-2a1076a2bf2b admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:09.218280 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a5e25d-b2df-4368-9877-af398341f218 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:09.219100 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a5e25d-b2df-4368-9877-af398341f218 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:09.219243 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a5e25d-b2df-4368-9877-af398341f218 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:09.219469 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a5e25d-b2df-4368-9877-af398341f218 admin admin] PATH_INFO: `/v3/projects/17117d29c70040e28166c64f1fb10541` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:09.220819 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a5e25d-b2df-4368-9877-af398341f218 admin admin] RBAC: Authorizing `identity:delete_project(project_id=17117d29c70040e28166c64f1fb10541)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:09.222950 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a5e25d-b2df-4368-9877-af398341f218 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:09.252282 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f7090a-802b-4161-8576-b56245a5fb1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:09.253118 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f7090a-802b-4161-8576-b56245a5fb1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:09.253811 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f7090a-802b-4161-8576-b56245a5fb1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:09.253811 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f7090a-802b-4161-8576-b56245a5fb1d admin admin] PATH_INFO: `/v3/projects/d834ee64b8bb450c9ecd872b5c194862` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:09.255188 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f7090a-802b-4161-8576-b56245a5fb1d admin admin] RBAC: Authorizing `identity:delete_project(project_id=d834ee64b8bb450c9ecd872b5c194862)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:09.257380 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f7090a-802b-4161-8576-b56245a5fb1d admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:09.281981 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-22a5e25d-b2df-4368-9877-af398341f218 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '17117d29c70040e28166c64f1fb10541', 'request_id': 'req-22a5e25d-b2df-4368-9877-af398341f218'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:09.293982 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-22a5e25d-b2df-4368-9877-af398341f218 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '17117d29c70040e28166c64f1fb10541', 'request_id': 'req-22a5e25d-b2df-4368-9877-af398341f218'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:09.302633 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-22a5e25d-b2df-4368-9877-af398341f218 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '17117d29c70040e28166c64f1fb10541', 'request_id': 'req-22a5e25d-b2df-4368-9877-af398341f218'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:09.318480 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-49f7090a-802b-4161-8576-b56245a5fb1d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd834ee64b8bb450c9ecd872b5c194862', 'request_id': 'req-49f7090a-802b-4161-8576-b56245a5fb1d'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:09.322558 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-22a5e25d-b2df-4368-9877-af398341f218 admin admin] The token cache is being invalidate because project 17117d29c70040e28166c64f1fb10541 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81758) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:41:09.323011 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-22a5e25d-b2df-4368-9877-af398341f218 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:09.323776 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-22a5e25d-b2df-4368-9877-af398341f218 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:09.324403 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-49f7090a-802b-4161-8576-b56245a5fb1d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd834ee64b8bb450c9ecd872b5c194862', 'request_id': 'req-49f7090a-802b-4161-8576-b56245a5fb1d'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:09.328651 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 954/1906] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 17:41:09 2022] DELETE /identity/v3/projects/17117d29c70040e28166c64f1fb10541 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:41:09.328651 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a5e25d-b2df-4368-9877-af398341f218 admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:09.328651 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-22a5e25d-b2df-4368-9877-af398341f218 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:09.334320 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-49f7090a-802b-4161-8576-b56245a5fb1d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd834ee64b8bb450c9ecd872b5c194862', 'request_id': 'req-49f7090a-802b-4161-8576-b56245a5fb1d'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:09.349432 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-49f7090a-802b-4161-8576-b56245a5fb1d admin admin] The token cache is being invalidate because project d834ee64b8bb450c9ecd872b5c194862 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81759) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:41:09.349873 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-49f7090a-802b-4161-8576-b56245a5fb1d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:09.350596 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-49f7090a-802b-4161-8576-b56245a5fb1d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:09.352426 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 953/1907] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 17:41:09 2022] DELETE /identity/v3/projects/d834ee64b8bb450c9ecd872b5c194862 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:41:09.353530 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed193ad8-0972-4ed4-a03c-d2f73fbef141 None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:09.353910 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed193ad8-0972-4ed4-a03c-d2f73fbef141 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:09.354280 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed193ad8-0972-4ed4-a03c-d2f73fbef141 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:09.383184 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-ed193ad8-0972-4ed4-a03c-d2f73fbef141 None None] MFA Rules not processed for user `7bf0ab87f39d4ae0aa21f87faff1ed6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:41:09.384568 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-ed193ad8-0972-4ed4-a03c-d2f73fbef141 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:09.400085 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:41:09.402414 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a5e25d-b2df-4368-9877-af398341f218 admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:09.482390 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a18f1ac-0ccf-43f4-bf63-c4a6e1411bba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:09.483982 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a18f1ac-0ccf-43f4-bf63-c4a6e1411bba admin admin] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:09.485009 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a18f1ac-0ccf-43f4-bf63-c4a6e1411bba admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:09.485806 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a18f1ac-0ccf-43f4-bf63-c4a6e1411bba admin admin] PATH_INFO: `/v3/users` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:09.487294 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a18f1ac-0ccf-43f4-bf63-c4a6e1411bba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:09.489532 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a18f1ac-0ccf-43f4-bf63-c4a6e1411bba admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:09.493229 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 954/1908] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:41:09 2022] POST /identity/v3/auth/tokens => generated 3351 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:41:09.494527 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e1faf1-9d05-4eff-8920-0c57db7690fa None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:09.494814 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e1faf1-9d05-4eff-8920-0c57db7690fa None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:09.495092 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e1faf1-9d05-4eff-8920-0c57db7690fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:09.507866 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:41:09.526220 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-b3e1faf1-9d05-4eff-8920-0c57db7690fa None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:41:09.527499 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-b3e1faf1-9d05-4eff-8920-0c57db7690fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:09.531400 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 955/1909] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 17:41:09 2022] POST /identity/v3/users => generated 293 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:41:09.532582 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952c1f8e-330c-41e9-9cc7-19036d864849 None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:09.533534 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952c1f8e-330c-41e9-9cc7-19036d864849 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:09.533731 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952c1f8e-330c-41e9-9cc7-19036d864849 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:09.539955 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 955/1910] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:41:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:41:09.540838 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e1faf1-9d05-4eff-8920-0c57db7690fa None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:09.542143 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-b3e1faf1-9d05-4eff-8920-0c57db7690fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:09.557663 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-952c1f8e-330c-41e9-9cc7-19036d864849 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:41:09.559545 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-952c1f8e-330c-41e9-9cc7-19036d864849 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:09.569248 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 956/1911] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:41:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:41:09.570084 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952c1f8e-330c-41e9-9cc7-19036d864849 None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:09.571373 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-952c1f8e-330c-41e9-9cc7-19036d864849 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:09.583509 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952c1f8e-330c-41e9-9cc7-19036d864849 None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:09.588077 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c842dc7-aff7-4a76-b218-0a992ed2d85c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:09.588717 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c842dc7-aff7-4a76-b218-0a992ed2d85c admin admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:09.588977 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c842dc7-aff7-4a76-b218-0a992ed2d85c admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:09.589253 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c842dc7-aff7-4a76-b218-0a992ed2d85c admin admin] PATH_INFO: `/v3/roles` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:09.589890 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c842dc7-aff7-4a76-b218-0a992ed2d85c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:09.591479 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c842dc7-aff7-4a76-b218-0a992ed2d85c admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:09.605020 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 957/1912] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:41:09 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:41:09.605183 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e1faf1-9d05-4eff-8920-0c57db7690fa None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:09.605277 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200f40d3-0e1a-4786-8ddb-be96e285e6b9 None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:09.605277 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200f40d3-0e1a-4786-8ddb-be96e285e6b9 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:09.605277 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200f40d3-0e1a-4786-8ddb-be96e285e6b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:09.608906 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f7fc79-7ac6-408f-a4ad-5f7bc5fa7688 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59be342541264942ad87f60ff4de6c8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:09.609616 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f7fc79-7ac6-408f-a4ad-5f7bc5fa7688 service cinder] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:09.609970 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f7fc79-7ac6-408f-a4ad-5f7bc5fa7688 service cinder] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:09.610319 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f7fc79-7ac6-408f-a4ad-5f7bc5fa7688 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:09.620184 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f7fc79-7ac6-408f-a4ad-5f7bc5fa7688 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:09.621649 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f7fc79-7ac6-408f-a4ad-5f7bc5fa7688 service cinder] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:09.623330 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-200f40d3-0e1a-4786-8ddb-be96e285e6b9 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:41:09.624512 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-200f40d3-0e1a-4786-8ddb-be96e285e6b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:09.631319 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 956/1913] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:41:09 2022] GET /identity/v3/auth/tokens => generated 3351 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:41:09.635682 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 958/1914] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:41:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:41:09.635682 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200f40d3-0e1a-4786-8ddb-be96e285e6b9 None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:09.636416 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06289553-1c70-4276-81fe-7e96bf4bc366 None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:09.636878 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06289553-1c70-4276-81fe-7e96bf4bc366 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:09.637725 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06289553-1c70-4276-81fe-7e96bf4bc366 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:09.644507 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200f40d3-0e1a-4786-8ddb-be96e285e6b9 None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:09.649283 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b395a4-1e45-43e1-b9e0-f4dce7905e4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:09.649880 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b395a4-1e45-43e1-b9e0-f4dce7905e4c admin admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:09.650184 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b395a4-1e45-43e1-b9e0-f4dce7905e4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:09.650561 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b395a4-1e45-43e1-b9e0-f4dce7905e4c admin admin] PATH_INFO: `/v3/roles` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:09.651270 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b395a4-1e45-43e1-b9e0-f4dce7905e4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:09.652820 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b395a4-1e45-43e1-b9e0-f4dce7905e4c admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:09.658978 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-06289553-1c70-4276-81fe-7e96bf4bc366 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:41:09.660173 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-06289553-1c70-4276-81fe-7e96bf4bc366 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:09.663903 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 959/1915] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:41:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:41:09.667876 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b395a4-1e45-43e1-b9e0-f4dce7905e4c admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:09.668700 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 957/1916] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:41:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:41:09.674271 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06289553-1c70-4276-81fe-7e96bf4bc366 None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:09.679981 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b395a4-1e45-43e1-b9e0-f4dce7905e4c admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:09.684511 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afd2a36-19d7-4abe-992d-48e6a6925461 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:09.685143 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afd2a36-19d7-4abe-992d-48e6a6925461 admin admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:09.685422 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afd2a36-19d7-4abe-992d-48e6a6925461 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:09.685698 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afd2a36-19d7-4abe-992d-48e6a6925461 admin admin] PATH_INFO: `/v3/domains` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:09.685784 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06289553-1c70-4276-81fe-7e96bf4bc366 None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:09.686363 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afd2a36-19d7-4abe-992d-48e6a6925461 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:41:09.686634 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afd2a36-19d7-4abe-992d-48e6a6925461 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:09.688975 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afd2a36-19d7-4abe-992d-48e6a6925461 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:09.690693 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf9a6b6-62c1-4916-986a-d26398b261c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:09.691353 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf9a6b6-62c1-4916-986a-d26398b261c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:09.691628 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf9a6b6-62c1-4916-986a-d26398b261c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:09.691907 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf9a6b6-62c1-4916-986a-d26398b261c5 admin admin] PATH_INFO: `/v3/projects/6f2bd28d060349188c072d5b1ce847b1/users/8b30c795a59641218c9d3626edb34b8b/roles/02476a5b82eb4b188c18a5d706a3ef68` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:09.703118 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 960/1917] 10.222.0.45 () {58 vars in 1153 bytes} [Wed Aug 24 17:41:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:41:09.703944 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afd2a36-19d7-4abe-992d-48e6a6925461 admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:09.707950 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf9a6b6-62c1-4916-986a-d26398b261c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f2bd28d060349188c072d5b1ce847b1, user_id=8b30c795a59641218c9d3626edb34b8b, role_id=02476a5b82eb4b188c18a5d706a3ef68)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:09.710727 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf9a6b6-62c1-4916-986a-d26398b261c5 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:09.716865 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afd2a36-19d7-4abe-992d-48e6a6925461 admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:09.725256 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a1ffec-2211-4529-8e42-c0ad52868131 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:09.735120 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a1ffec-2211-4529-8e42-c0ad52868131 admin admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:09.735316 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 958/1918] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 17:41:09 2022] PUT /identity/v3/projects/6f2bd28d060349188c072d5b1ce847b1/users/8b30c795a59641218c9d3626edb34b8b/roles/02476a5b82eb4b188c18a5d706a3ef68 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:41:09.735432 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a1ffec-2211-4529-8e42-c0ad52868131 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:09.735432 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a1ffec-2211-4529-8e42-c0ad52868131 admin admin] PATH_INFO: `/v3/domains` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:09.735590 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf9a6b6-62c1-4916-986a-d26398b261c5 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:09.735675 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a1ffec-2211-4529-8e42-c0ad52868131 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:41:09.735675 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a1ffec-2211-4529-8e42-c0ad52868131 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:09.735675 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a1ffec-2211-4529-8e42-c0ad52868131 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:09.743055 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 961/1919] 10.222.0.45 () {58 vars in 1153 bytes} [Wed Aug 24 17:41:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:41:09.744365 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25cae4b-94d1-4c09-984e-ea409a69f524 None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:09.744760 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25cae4b-94d1-4c09-984e-ea409a69f524 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:09.745100 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25cae4b-94d1-4c09-984e-ea409a69f524 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:09.779526 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-e25cae4b-94d1-4c09-984e-ea409a69f524 None None] MFA Rules not processed for user `8b30c795a59641218c9d3626edb34b8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:41:09.781331 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-e25cae4b-94d1-4c09-984e-ea409a69f524 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:09.822807 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf9a6b6-62c1-4916-986a-d26398b261c5 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:09.827799 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95ef016-51bb-41de-a95d-d916a84fdcab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:09.828409 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95ef016-51bb-41de-a95d-d916a84fdcab admin admin] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:09.828745 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95ef016-51bb-41de-a95d-d916a84fdcab admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:09.829424 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95ef016-51bb-41de-a95d-d916a84fdcab admin admin] PATH_INFO: `/v3/projects` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:09.829758 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95ef016-51bb-41de-a95d-d916a84fdcab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:09.831658 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95ef016-51bb-41de-a95d-d916a84fdcab admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:09.857301 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 962/1920] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:41:09 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:41:09.861922 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25cae4b-94d1-4c09-984e-ea409a69f524 None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:09.862057 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 959/1921] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 17:41:09 2022] POST /identity/v3/projects => generated 379 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:41:09.862057 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95ef016-51bb-41de-a95d-d916a84fdcab admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:09.865039 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:41:09.871315 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25cae4b-94d1-4c09-984e-ea409a69f524 None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:09.917582 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95ef016-51bb-41de-a95d-d916a84fdcab admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:09.921313 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869ce6da-9aad-4840-8eda-f7c96792f55c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59be342541264942ad87f60ff4de6c8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:09.921921 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869ce6da-9aad-4840-8eda-f7c96792f55c service cinder] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:09.922203 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869ce6da-9aad-4840-8eda-f7c96792f55c service cinder] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:09.922479 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869ce6da-9aad-4840-8eda-f7c96792f55c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:09.924289 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-869ce6da-9aad-4840-8eda-f7c96792f55c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:09.927657 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f645c97-2d9d-4ea3-817a-33416e7b5bab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:09.928292 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f645c97-2d9d-4ea3-817a-33416e7b5bab admin admin] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:09.928591 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f645c97-2d9d-4ea3-817a-33416e7b5bab admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:09.928869 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f645c97-2d9d-4ea3-817a-33416e7b5bab admin admin] PATH_INFO: `/v3/projects` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:09.929510 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f645c97-2d9d-4ea3-817a-33416e7b5bab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:09.931397 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f645c97-2d9d-4ea3-817a-33416e7b5bab admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:09.958087 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 963/1922] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 17:41:09 2022] POST /identity/v3/projects => generated 385 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:41:09.959023 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f645c97-2d9d-4ea3-817a-33416e7b5bab admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:09.963092 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:41:09.987683 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869ce6da-9aad-4840-8eda-f7c96792f55c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:09.989173 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869ce6da-9aad-4840-8eda-f7c96792f55c service cinder] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:09.998017 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 960/1923] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:41:09 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:41:09.998850 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869ce6da-9aad-4840-8eda-f7c96792f55c service cinder] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:10.010012 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869ce6da-9aad-4840-8eda-f7c96792f55c service cinder] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:10.028511 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f645c97-2d9d-4ea3-817a-33416e7b5bab admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:10.033743 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c1c435-d57a-4931-ad9b-bbaafb2b8f25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:10.034357 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c1c435-d57a-4931-ad9b-bbaafb2b8f25 admin admin] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:10.035122 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c1c435-d57a-4931-ad9b-bbaafb2b8f25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:10.035471 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c1c435-d57a-4931-ad9b-bbaafb2b8f25 admin admin] PATH_INFO: `/v3/users` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:10.036224 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c1c435-d57a-4931-ad9b-bbaafb2b8f25 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:10.038784 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c1c435-d57a-4931-ad9b-bbaafb2b8f25 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:10.064989 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 964/1924] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 17:41:09 2022] POST /identity/v3/users => generated 290 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:41:10.066454 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c1c435-d57a-4931-ad9b-bbaafb2b8f25 admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:10.077251 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ff683d-dc81-45f3-abf4-ff403941db8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:10.077559 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c1c435-d57a-4931-ad9b-bbaafb2b8f25 admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:10.078432 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ff683d-dc81-45f3-abf4-ff403941db8b admin admin] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:10.078805 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ff683d-dc81-45f3-abf4-ff403941db8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:10.079174 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ff683d-dc81-45f3-abf4-ff403941db8b admin admin] PATH_INFO: `/v3/projects` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:10.079952 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ff683d-dc81-45f3-abf4-ff403941db8b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:10.082212 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ff683d-dc81-45f3-abf4-ff403941db8b admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:10.082433 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e951d11-bacb-4615-b4fd-ec4b943a2ae1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:10.083166 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e951d11-bacb-4615-b4fd-ec4b943a2ae1 admin admin] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:10.083537 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e951d11-bacb-4615-b4fd-ec4b943a2ae1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:10.083816 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e951d11-bacb-4615-b4fd-ec4b943a2ae1 admin admin] PATH_INFO: `/v3/users` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:10.084496 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e951d11-bacb-4615-b4fd-ec4b943a2ae1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:10.086687 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e951d11-bacb-4615-b4fd-ec4b943a2ae1 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:10.116203 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 965/1925] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 17:41:10 2022] POST /identity/v3/users => generated 293 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:41:10.117509 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e951d11-bacb-4615-b4fd-ec4b943a2ae1 admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:10.122846 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 961/1926] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 17:41:09 2022] POST /identity/v3/projects => generated 387 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:41:10.123583 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ff683d-dc81-45f3-abf4-ff403941db8b admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:10.132271 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e951d11-bacb-4615-b4fd-ec4b943a2ae1 admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:10.200079 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6472baa6-ca90-4b96-a8e9-54685254f39d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:10.200766 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6472baa6-ca90-4b96-a8e9-54685254f39d admin admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:10.201049 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6472baa6-ca90-4b96-a8e9-54685254f39d admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:10.201327 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6472baa6-ca90-4b96-a8e9-54685254f39d admin admin] PATH_INFO: `/v3/roles` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:10.201983 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6472baa6-ca90-4b96-a8e9-54685254f39d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:10.203664 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6472baa6-ca90-4b96-a8e9-54685254f39d admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:10.206577 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:41:10.220471 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 966/1927] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:41:10 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:41:10.220471 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6472baa6-ca90-4b96-a8e9-54685254f39d admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:10.231294 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6472baa6-ca90-4b96-a8e9-54685254f39d admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:10.237869 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61971b69-5328-4ccd-bafa-7a04179e17bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:10.238484 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61971b69-5328-4ccd-bafa-7a04179e17bc admin admin] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:10.238769 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61971b69-5328-4ccd-bafa-7a04179e17bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:10.239047 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61971b69-5328-4ccd-bafa-7a04179e17bc admin admin] PATH_INFO: `/v3/users` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:10.239831 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61971b69-5328-4ccd-bafa-7a04179e17bc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:10.241970 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61971b69-5328-4ccd-bafa-7a04179e17bc admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:10.243659 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ff683d-dc81-45f3-abf4-ff403941db8b admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:10.253109 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fefdb4-67eb-4f26-886b-c3f971edffa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:10.253772 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fefdb4-67eb-4f26-886b-c3f971edffa0 admin admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:10.254027 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fefdb4-67eb-4f26-886b-c3f971edffa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:10.254297 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fefdb4-67eb-4f26-886b-c3f971edffa0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:10.254975 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fefdb4-67eb-4f26-886b-c3f971edffa0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:10.256753 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fefdb4-67eb-4f26-886b-c3f971edffa0 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:10.267745 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 967/1928] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 17:41:10 2022] POST /identity/v3/users => generated 300 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:41:10.271620 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 962/1929] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:41:10 2022] GET /identity/v3/roles => generated 1420 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:41:10.271804 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61971b69-5328-4ccd-bafa-7a04179e17bc admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:10.272572 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fefdb4-67eb-4f26-886b-c3f971edffa0 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:10.282693 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fefdb4-67eb-4f26-886b-c3f971edffa0 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:10.283480 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61971b69-5328-4ccd-bafa-7a04179e17bc admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:10.288796 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2418d9a3-f16e-4c5e-93b4-2b8239d5127a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:10.289342 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b96173-5bd3-432b-a53e-41540cff7be0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:10.289710 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2418d9a3-f16e-4c5e-93b4-2b8239d5127a admin admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:10.290093 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2418d9a3-f16e-4c5e-93b4-2b8239d5127a admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:10.290224 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b96173-5bd3-432b-a53e-41540cff7be0 admin admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:10.290456 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b96173-5bd3-432b-a53e-41540cff7be0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:10.290621 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2418d9a3-f16e-4c5e-93b4-2b8239d5127a admin admin] PATH_INFO: `/v3/roles` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:10.290721 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b96173-5bd3-432b-a53e-41540cff7be0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:10.291381 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b96173-5bd3-432b-a53e-41540cff7be0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:10.291542 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2418d9a3-f16e-4c5e-93b4-2b8239d5127a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:10.293078 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b96173-5bd3-432b-a53e-41540cff7be0 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:10.293493 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2418d9a3-f16e-4c5e-93b4-2b8239d5127a admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:10.307993 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 968/1930] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:41:10 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:41:10.311135 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 963/1931] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:41:10 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:41:10.312247 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b96173-5bd3-432b-a53e-41540cff7be0 admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:10.312872 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2418d9a3-f16e-4c5e-93b4-2b8239d5127a admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:10.324003 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2418d9a3-f16e-4c5e-93b4-2b8239d5127a admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:10.324307 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b96173-5bd3-432b-a53e-41540cff7be0 admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:10.332441 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e8842d-9431-467f-a073-89998481f0e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:10.333227 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e8842d-9431-467f-a073-89998481f0e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:10.333514 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e8842d-9431-467f-a073-89998481f0e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:10.333808 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e8842d-9431-467f-a073-89998481f0e7 admin admin] PATH_INFO: `/v3/projects/d65e1cd424f24093976f978e5b137e32/users/7b322976a1694e2b9bcbd10b0f5b8fe6/roles/e411ad9eb96e4e8d85b59b5b700ada7b` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:10.343143 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81365a45-d5ec-4294-99d9-30cb6d1b4743 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:10.343889 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81365a45-d5ec-4294-99d9-30cb6d1b4743 admin admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:10.347202 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81365a45-d5ec-4294-99d9-30cb6d1b4743 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:10.347685 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81365a45-d5ec-4294-99d9-30cb6d1b4743 admin admin] PATH_INFO: `/v3/roles` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:10.348642 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81365a45-d5ec-4294-99d9-30cb6d1b4743 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:10.350474 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81365a45-d5ec-4294-99d9-30cb6d1b4743 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:10.359038 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e8842d-9431-467f-a073-89998481f0e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d65e1cd424f24093976f978e5b137e32, user_id=7b322976a1694e2b9bcbd10b0f5b8fe6, role_id=e411ad9eb96e4e8d85b59b5b700ada7b)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:10.362878 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e8842d-9431-467f-a073-89998481f0e7 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:10.374599 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 964/1932] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:41:10 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:41:10.376661 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81365a45-d5ec-4294-99d9-30cb6d1b4743 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:10.385567 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 969/1933] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 17:41:10 2022] PUT /identity/v3/projects/d65e1cd424f24093976f978e5b137e32/users/7b322976a1694e2b9bcbd10b0f5b8fe6/roles/e411ad9eb96e4e8d85b59b5b700ada7b => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:41:10.386858 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e8842d-9431-467f-a073-89998481f0e7 admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:10.398210 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81365a45-d5ec-4294-99d9-30cb6d1b4743 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:10.418647 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:41:10.490825 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0cb483-42c0-44d3-aed3-19ed77e8ab49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:10.491515 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0cb483-42c0-44d3-aed3-19ed77e8ab49 admin admin] REQUEST_METHOD: `PUT` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:10.491793 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0cb483-42c0-44d3-aed3-19ed77e8ab49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:10.492115 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0cb483-42c0-44d3-aed3-19ed77e8ab49 admin admin] PATH_INFO: `/v3/projects/87a610c98b9b48bd96d2954604bf3078/users/af6710bfe62d4ba29dc48e2f7aba9ec5/roles/02476a5b82eb4b188c18a5d706a3ef68` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:10.504054 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e8842d-9431-467f-a073-89998481f0e7 admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:10.514079 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0cb483-42c0-44d3-aed3-19ed77e8ab49 admin admin] RBAC: Authorizing `identity:create_grant(project_id=87a610c98b9b48bd96d2954604bf3078, user_id=af6710bfe62d4ba29dc48e2f7aba9ec5, role_id=02476a5b82eb4b188c18a5d706a3ef68)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:10.514878 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ccad82-a30e-4cc3-82d8-83b7ca8e6da0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:10.517028 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0cb483-42c0-44d3-aed3-19ed77e8ab49 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:10.517890 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ccad82-a30e-4cc3-82d8-83b7ca8e6da0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:10.518358 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ccad82-a30e-4cc3-82d8-83b7ca8e6da0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:10.518736 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ccad82-a30e-4cc3-82d8-83b7ca8e6da0 admin admin] PATH_INFO: `/v3/projects/b854a0fd31b24efaab31c07cde312c22/users/cd73655291f0459ca6f7677e90a9e073/roles/02476a5b82eb4b188c18a5d706a3ef68` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:10.540082 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 965/1934] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 17:41:10 2022] PUT /identity/v3/projects/87a610c98b9b48bd96d2954604bf3078/users/af6710bfe62d4ba29dc48e2f7aba9ec5/roles/02476a5b82eb4b188c18a5d706a3ef68 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:41:10.541524 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0cb483-42c0-44d3-aed3-19ed77e8ab49 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:10.557693 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ccad82-a30e-4cc3-82d8-83b7ca8e6da0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b854a0fd31b24efaab31c07cde312c22, user_id=cd73655291f0459ca6f7677e90a9e073, role_id=02476a5b82eb4b188c18a5d706a3ef68)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:10.578733 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ccad82-a30e-4cc3-82d8-83b7ca8e6da0 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:10.598435 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 970/1935] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 17:41:10 2022] PUT /identity/v3/projects/b854a0fd31b24efaab31c07cde312c22/users/cd73655291f0459ca6f7677e90a9e073/roles/02476a5b82eb4b188c18a5d706a3ef68 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:41:10.599856 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb95f595-0ef3-4dac-856d-5900a63b46ca None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:10.600386 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb95f595-0ef3-4dac-856d-5900a63b46ca None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:10.600887 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb95f595-0ef3-4dac-856d-5900a63b46ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:10.626773 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-eb95f595-0ef3-4dac-856d-5900a63b46ca None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:41:10.628124 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-eb95f595-0ef3-4dac-856d-5900a63b46ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:10.644346 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0cb483-42c0-44d3-aed3-19ed77e8ab49 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:10.695825 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:41:10.710748 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 971/1936] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:41:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:41:10.712020 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6099a11a-31ce-466e-b319-d1c47189ff83 None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:10.712377 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6099a11a-31ce-466e-b319-d1c47189ff83 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:10.712758 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6099a11a-31ce-466e-b319-d1c47189ff83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:10.721322 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fd631b-3c0e-4f23-bc04-f272d767f600 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:10.722426 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fd631b-3c0e-4f23-bc04-f272d767f600 admin admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:10.722566 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fd631b-3c0e-4f23-bc04-f272d767f600 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:10.722808 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fd631b-3c0e-4f23-bc04-f272d767f600 admin admin] PATH_INFO: `/v3/roles` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:10.723561 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fd631b-3c0e-4f23-bc04-f272d767f600 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:10.725326 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fd631b-3c0e-4f23-bc04-f272d767f600 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:10.735704 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 966/1937] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:41:10 2022] GET /identity/v3/roles => generated 1420 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:41:10.736673 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fd631b-3c0e-4f23-bc04-f272d767f600 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:10.738351 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-77fd631b-3c0e-4f23-bc04-f272d767f600 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:10.742850 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-6099a11a-31ce-466e-b319-d1c47189ff83 None None] MFA Rules not processed for user `cd73655291f0459ca6f7677e90a9e073`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:41:10.744022 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-6099a11a-31ce-466e-b319-d1c47189ff83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:10.805522 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fd631b-3c0e-4f23-bc04-f272d767f600 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:10.809353 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99b6a98-cc80-42b3-8a19-853ef18da54b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43ad98a250a144099ccd18378aedd4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:10.809958 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99b6a98-cc80-42b3-8a19-853ef18da54b service neutron] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:10.810237 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99b6a98-cc80-42b3-8a19-853ef18da54b service neutron] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:10.810515 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99b6a98-cc80-42b3-8a19-853ef18da54b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:10.819399 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99b6a98-cc80-42b3-8a19-853ef18da54b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:10.820871 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99b6a98-cc80-42b3-8a19-853ef18da54b service neutron] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:10.823957 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 972/1938] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:41:10 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:41:10.824809 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6099a11a-31ce-466e-b319-d1c47189ff83 None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:10.831907 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 967/1939] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:41:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:41:10.832741 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99b6a98-cc80-42b3-8a19-853ef18da54b service neutron] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:10.835704 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6099a11a-31ce-466e-b319-d1c47189ff83 None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:10.840374 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a593a6-ef39-4b87-a109-4828b3845474 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:10.841067 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a593a6-ef39-4b87-a109-4828b3845474 admin admin] REQUEST_METHOD: `PUT` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:10.841351 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a593a6-ef39-4b87-a109-4828b3845474 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:10.841627 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a593a6-ef39-4b87-a109-4828b3845474 admin admin] PATH_INFO: `/v3/projects/d65e1cd424f24093976f978e5b137e32/users/7b322976a1694e2b9bcbd10b0f5b8fe6/roles/e411ad9eb96e4e8d85b59b5b700ada7b` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:10.842284 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99b6a98-cc80-42b3-8a19-853ef18da54b service neutron] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:10.843624 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a593a6-ef39-4b87-a109-4828b3845474 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d65e1cd424f24093976f978e5b137e32, user_id=7b322976a1694e2b9bcbd10b0f5b8fe6, role_id=e411ad9eb96e4e8d85b59b5b700ada7b)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:10.847192 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a593a6-ef39-4b87-a109-4828b3845474 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:10.847369 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb18cc95-5dba-4311-90df-21f4d49651a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:10.847980 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb18cc95-5dba-4311-90df-21f4d49651a9 admin admin] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:10.848263 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb18cc95-5dba-4311-90df-21f4d49651a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:10.848586 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb18cc95-5dba-4311-90df-21f4d49651a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:10.849177 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb18cc95-5dba-4311-90df-21f4d49651a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:10.851055 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb18cc95-5dba-4311-90df-21f4d49651a9 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:10.860598 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 973/1940] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 17:41:10 2022] PUT /identity/v3/projects/d65e1cd424f24093976f978e5b137e32/users/7b322976a1694e2b9bcbd10b0f5b8fe6/roles/e411ad9eb96e4e8d85b59b5b700ada7b => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:41:10.870471 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75a3ffa-f1f0-4032-88c7-76da3bf9ca74 None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:10.870758 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75a3ffa-f1f0-4032-88c7-76da3bf9ca74 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:10.871035 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75a3ffa-f1f0-4032-88c7-76da3bf9ca74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:10.884309 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 968/1941] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 17:41:10 2022] POST /identity/v3/projects => generated 387 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:41:10.886144 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb18cc95-5dba-4311-90df-21f4d49651a9 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:10.901774 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-f75a3ffa-f1f0-4032-88c7-76da3bf9ca74 None None] MFA Rules not processed for user `7b322976a1694e2b9bcbd10b0f5b8fe6`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:41:10.903122 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-f75a3ffa-f1f0-4032-88c7-76da3bf9ca74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:10.910353 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:41:10.958795 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb18cc95-5dba-4311-90df-21f4d49651a9 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:10.964506 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b064bdd-5f27-4546-beed-9ca4e93a6549 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:10.965123 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b064bdd-5f27-4546-beed-9ca4e93a6549 admin admin] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:10.965405 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b064bdd-5f27-4546-beed-9ca4e93a6549 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:10.965717 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b064bdd-5f27-4546-beed-9ca4e93a6549 admin admin] PATH_INFO: `/v3/users` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:10.966335 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b064bdd-5f27-4546-beed-9ca4e93a6549 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:10.968291 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b064bdd-5f27-4546-beed-9ca4e93a6549 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:10.993618 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 969/1942] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 17:41:10 2022] POST /identity/v3/users => generated 300 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:41:10.997243 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b064bdd-5f27-4546-beed-9ca4e93a6549 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:10.998151 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 974/1943] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:41:10 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:41:11.009441 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75a3ffa-f1f0-4032-88c7-76da3bf9ca74 None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:11.010910 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b064bdd-5f27-4546-beed-9ca4e93a6549 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:11.017960 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0397aeb-4fa7-4179-ae8b-504de3ef7ed2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:11.018556 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0397aeb-4fa7-4179-ae8b-504de3ef7ed2 admin admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:11.018909 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0397aeb-4fa7-4179-ae8b-504de3ef7ed2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:11.019222 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0397aeb-4fa7-4179-ae8b-504de3ef7ed2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:11.019874 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0397aeb-4fa7-4179-ae8b-504de3ef7ed2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:11.021540 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0397aeb-4fa7-4179-ae8b-504de3ef7ed2 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:11.033658 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 970/1944] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:41:10 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:41:11.037566 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0397aeb-4fa7-4179-ae8b-504de3ef7ed2 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:11.052254 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0397aeb-4fa7-4179-ae8b-504de3ef7ed2 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:11.060942 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fd01ad-6061-4d03-8ca7-2f071280ded0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:11.061806 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fd01ad-6061-4d03-8ca7-2f071280ded0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:11.062157 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fd01ad-6061-4d03-8ca7-2f071280ded0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:11.062495 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fd01ad-6061-4d03-8ca7-2f071280ded0 admin admin] PATH_INFO: `/v3/projects/17dd8d2e584a4cb69bf4ef815df8373c/users/bf4d581dffbe43dc8543986d5394e414/roles/e411ad9eb96e4e8d85b59b5b700ada7b` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:11.081063 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fd01ad-6061-4d03-8ca7-2f071280ded0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=17dd8d2e584a4cb69bf4ef815df8373c, user_id=bf4d581dffbe43dc8543986d5394e414, role_id=e411ad9eb96e4e8d85b59b5b700ada7b)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:11.084017 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fd01ad-6061-4d03-8ca7-2f071280ded0 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:11.088766 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75a3ffa-f1f0-4032-88c7-76da3bf9ca74 None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:11.093080 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b83af1-200b-47b8-a2e2-96e19f98a72b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59be342541264942ad87f60ff4de6c8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:11.093606 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b83af1-200b-47b8-a2e2-96e19f98a72b service cinder] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:11.093899 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b83af1-200b-47b8-a2e2-96e19f98a72b service cinder] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:11.094157 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b83af1-200b-47b8-a2e2-96e19f98a72b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:11.103550 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b83af1-200b-47b8-a2e2-96e19f98a72b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:11.103715 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 971/1945] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 17:41:11 2022] PUT /identity/v3/projects/17dd8d2e584a4cb69bf4ef815df8373c/users/bf4d581dffbe43dc8543986d5394e414/roles/e411ad9eb96e4e8d85b59b5b700ada7b => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:41:11.105029 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b83af1-200b-47b8-a2e2-96e19f98a72b service cinder] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:11.107452 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fd01ad-6061-4d03-8ca7-2f071280ded0 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:11.115990 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 975/1946] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:41:11 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:41:11.130247 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:41:11.191401 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fd01ad-6061-4d03-8ca7-2f071280ded0 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:11.214910 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6703bd3-7d19-4a5b-8555-fd4ed2d78eef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:11.215821 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6703bd3-7d19-4a5b-8555-fd4ed2d78eef admin admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:11.216197 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6703bd3-7d19-4a5b-8555-fd4ed2d78eef admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:11.216592 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6703bd3-7d19-4a5b-8555-fd4ed2d78eef admin admin] PATH_INFO: `/v3/roles` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:11.217517 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6703bd3-7d19-4a5b-8555-fd4ed2d78eef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:11.219507 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6703bd3-7d19-4a5b-8555-fd4ed2d78eef admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:11.237748 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 972/1947] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:41:11 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 17:41:11.242099 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b83af1-200b-47b8-a2e2-96e19f98a72b service cinder] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:11.257238 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b83af1-200b-47b8-a2e2-96e19f98a72b service cinder] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:11.263662 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c3e3af-d9ca-4f80-9594-b1ce6d31ed93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:11.264389 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c3e3af-d9ca-4f80-9594-b1ce6d31ed93 admin admin] REQUEST_METHOD: `PUT` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:11.264773 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c3e3af-d9ca-4f80-9594-b1ce6d31ed93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:11.265284 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c3e3af-d9ca-4f80-9594-b1ce6d31ed93 admin admin] PATH_INFO: `/v3/projects/17dd8d2e584a4cb69bf4ef815df8373c/users/bf4d581dffbe43dc8543986d5394e414/roles/e411ad9eb96e4e8d85b59b5b700ada7b` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:11.269073 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c3e3af-d9ca-4f80-9594-b1ce6d31ed93 admin admin] RBAC: Authorizing `identity:create_grant(project_id=17dd8d2e584a4cb69bf4ef815df8373c, user_id=bf4d581dffbe43dc8543986d5394e414, role_id=e411ad9eb96e4e8d85b59b5b700ada7b)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:11.274886 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c3e3af-d9ca-4f80-9594-b1ce6d31ed93 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:11.306537 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 976/1948] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 17:41:11 2022] PUT /identity/v3/projects/17dd8d2e584a4cb69bf4ef815df8373c/users/bf4d581dffbe43dc8543986d5394e414/roles/e411ad9eb96e4e8d85b59b5b700ada7b => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:41:11.308987 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92825d5c-39e4-4230-a5c9-e06cc7d61e78 None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:11.310296 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92825d5c-39e4-4230-a5c9-e06cc7d61e78 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:11.310729 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92825d5c-39e4-4230-a5c9-e06cc7d61e78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:11.345578 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-92825d5c-39e4-4230-a5c9-e06cc7d61e78 None None] MFA Rules not processed for user `bf4d581dffbe43dc8543986d5394e414`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:41:11.347017 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-92825d5c-39e4-4230-a5c9-e06cc7d61e78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:11.427341 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 973/1949] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:41:11 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:41:11.440870 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c3e3af-d9ca-4f80-9594-b1ce6d31ed93 admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:11.508291 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c3e3af-d9ca-4f80-9594-b1ce6d31ed93 admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:11.512284 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fc2b5c-870b-4891-a798-310e8ad6541c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59be342541264942ad87f60ff4de6c8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:11.513433 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fc2b5c-870b-4891-a798-310e8ad6541c service cinder] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:11.513719 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fc2b5c-870b-4891-a798-310e8ad6541c service cinder] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:11.514045 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fc2b5c-870b-4891-a798-310e8ad6541c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:11.528351 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fc2b5c-870b-4891-a798-310e8ad6541c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:11.530208 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fc2b5c-870b-4891-a798-310e8ad6541c service cinder] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:11.544827 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 977/1950] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:41:11 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:41:11.575570 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92825d5c-39e4-4230-a5c9-e06cc7d61e78 None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:11.782206 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92825d5c-39e4-4230-a5c9-e06cc7d61e78 None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:11.803240 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2627dd0-13c8-4aaf-a944-8b5bb89ec5ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:11.805061 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2627dd0-13c8-4aaf-a944-8b5bb89ec5ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:11.806073 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2627dd0-13c8-4aaf-a944-8b5bb89ec5ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:11.807710 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2627dd0-13c8-4aaf-a944-8b5bb89ec5ee admin admin] PATH_INFO: `/v3/users/cd73655291f0459ca6f7677e90a9e073` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:11.811866 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2627dd0-13c8-4aaf-a944-8b5bb89ec5ee admin admin] RBAC: Authorizing `identity:delete_user(user_id=cd73655291f0459ca6f7677e90a9e073)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:11.818695 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2627dd0-13c8-4aaf-a944-8b5bb89ec5ee admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:11.883862 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fc2b5c-870b-4891-a798-310e8ad6541c service cinder] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:11.885962 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-d1fc2b5c-870b-4891-a798-310e8ad6541c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:11.908545 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fc2b5c-870b-4891-a798-310e8ad6541c service cinder] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:11.919076 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083558a6-7cb6-473f-a7c7-82b1c86bff63 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59be342541264942ad87f60ff4de6c8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:11.920807 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:41:11.921086 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083558a6-7cb6-473f-a7c7-82b1c86bff63 service cinder] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:11.921366 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083558a6-7cb6-473f-a7c7-82b1c86bff63 service cinder] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:11.922337 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083558a6-7cb6-473f-a7c7-82b1c86bff63 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:11.928090 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-083558a6-7cb6-473f-a7c7-82b1c86bff63 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:11.933207 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-c2627dd0-13c8-4aaf-a944-8b5bb89ec5ee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cd73655291f0459ca6f7677e90a9e073'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:11.934572 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-c2627dd0-13c8-4aaf-a944-8b5bb89ec5ee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cd73655291f0459ca6f7677e90a9e073'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:11.953011 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-c2627dd0-13c8-4aaf-a944-8b5bb89ec5ee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cd73655291f0459ca6f7677e90a9e073'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:11.956653 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-c2627dd0-13c8-4aaf-a944-8b5bb89ec5ee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cd73655291f0459ca6f7677e90a9e073'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:11.973902 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-c2627dd0-13c8-4aaf-a944-8b5bb89ec5ee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cd73655291f0459ca6f7677e90a9e073'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:12.013074 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 974/1951] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 17:41:11 2022] DELETE /identity/v3/users/cd73655291f0459ca6f7677e90a9e073 => generated 0 bytes in 440 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:41:12.014318 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:41:12.018641 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a692ff-2601-4335-9d22-5e14356caee8 None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:12.018880 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a692ff-2601-4335-9d22-5e14356caee8 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:12.019234 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a692ff-2601-4335-9d22-5e14356caee8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:12.073172 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-03a692ff-2601-4335-9d22-5e14356caee8 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:41:12.075832 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-03a692ff-2601-4335-9d22-5e14356caee8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:12.086049 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083558a6-7cb6-473f-a7c7-82b1c86bff63 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:12.109753 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083558a6-7cb6-473f-a7c7-82b1c86bff63 service cinder] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:12.133686 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 978/1952] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:41:11 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 251 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:41:12.226685 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 975/1953] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:41:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:41:12.302213 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083558a6-7cb6-473f-a7c7-82b1c86bff63 service cinder] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:12.417089 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083558a6-7cb6-473f-a7c7-82b1c86bff63 service cinder] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:12.509854 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa7c484-2a8a-49b5-8e71-081170fbf3d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43ad98a250a144099ccd18378aedd4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:12.510108 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa7c484-2a8a-49b5-8e71-081170fbf3d3 service neutron] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:12.510661 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa7c484-2a8a-49b5-8e71-081170fbf3d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:12.511111 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa7c484-2a8a-49b5-8e71-081170fbf3d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:12.524135 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa7c484-2a8a-49b5-8e71-081170fbf3d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:12.526609 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa7c484-2a8a-49b5-8e71-081170fbf3d3 service neutron] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:12.542920 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 979/1954] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:41:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:41:13.030202 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a692ff-2601-4335-9d22-5e14356caee8 None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:13.033184 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-03a692ff-2601-4335-9d22-5e14356caee8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:13.055771 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a692ff-2601-4335-9d22-5e14356caee8 None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:13.061910 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05782620-e2df-4f27-a393-55e9d2182015 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:13.062744 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05782620-e2df-4f27-a393-55e9d2182015 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:13.063107 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05782620-e2df-4f27-a393-55e9d2182015 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:13.063441 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05782620-e2df-4f27-a393-55e9d2182015 admin admin] PATH_INFO: `/v3/projects/b854a0fd31b24efaab31c07cde312c22` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:13.065388 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05782620-e2df-4f27-a393-55e9d2182015 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b854a0fd31b24efaab31c07cde312c22)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:13.067538 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05782620-e2df-4f27-a393-55e9d2182015 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:13.128725 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-05782620-e2df-4f27-a393-55e9d2182015 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b854a0fd31b24efaab31c07cde312c22', 'request_id': 'req-05782620-e2df-4f27-a393-55e9d2182015'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:13.143829 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-05782620-e2df-4f27-a393-55e9d2182015 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b854a0fd31b24efaab31c07cde312c22', 'request_id': 'req-05782620-e2df-4f27-a393-55e9d2182015'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:13.151119 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-05782620-e2df-4f27-a393-55e9d2182015 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b854a0fd31b24efaab31c07cde312c22', 'request_id': 'req-05782620-e2df-4f27-a393-55e9d2182015'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:13.182310 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-05782620-e2df-4f27-a393-55e9d2182015 admin admin] The token cache is being invalidate because project b854a0fd31b24efaab31c07cde312c22 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81759) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:41:13.182611 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-05782620-e2df-4f27-a393-55e9d2182015 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:13.183285 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-05782620-e2df-4f27-a393-55e9d2182015 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:13.185165 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 976/1955] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 17:41:13 2022] DELETE /identity/v3/projects/b854a0fd31b24efaab31c07cde312c22 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:41:13.189868 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa7c484-2a8a-49b5-8e71-081170fbf3d3 service neutron] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:13.205404 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa7c484-2a8a-49b5-8e71-081170fbf3d3 service neutron] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:13.310678 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8f9f9b-e38c-4151-9432-cfdb8a521799 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:13.312713 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8f9f9b-e38c-4151-9432-cfdb8a521799 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:13.313779 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8f9f9b-e38c-4151-9432-cfdb8a521799 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:13.314247 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8f9f9b-e38c-4151-9432-cfdb8a521799 admin admin] PATH_INFO: `/v3/users/bf4d581dffbe43dc8543986d5394e414` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:13.316434 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8f9f9b-e38c-4151-9432-cfdb8a521799 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf4d581dffbe43dc8543986d5394e414)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:13.319895 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8f9f9b-e38c-4151-9432-cfdb8a521799 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:13.381559 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:41:13.396471 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-9b8f9f9b-e38c-4151-9432-cfdb8a521799 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf4d581dffbe43dc8543986d5394e414'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:13.397160 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-9b8f9f9b-e38c-4151-9432-cfdb8a521799 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf4d581dffbe43dc8543986d5394e414'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:13.413193 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-9b8f9f9b-e38c-4151-9432-cfdb8a521799 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf4d581dffbe43dc8543986d5394e414'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:13.415083 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-9b8f9f9b-e38c-4151-9432-cfdb8a521799 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf4d581dffbe43dc8543986d5394e414'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:13.446889 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-9b8f9f9b-e38c-4151-9432-cfdb8a521799 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf4d581dffbe43dc8543986d5394e414'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:13.467182 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 980/1956] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 17:41:13 2022] DELETE /identity/v3/users/bf4d581dffbe43dc8543986d5394e414 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:41:13.972095 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05782620-e2df-4f27-a393-55e9d2182015 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:13.973636 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-05782620-e2df-4f27-a393-55e9d2182015 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:13.989648 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05782620-e2df-4f27-a393-55e9d2182015 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:14.073772 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:41:14.099962 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23627d4-b8ee-408c-85d7-047109e13105 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:14.101438 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23627d4-b8ee-408c-85d7-047109e13105 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:14.101725 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23627d4-b8ee-408c-85d7-047109e13105 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:14.102001 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23627d4-b8ee-408c-85d7-047109e13105 admin admin] PATH_INFO: `/v3/projects/17dd8d2e584a4cb69bf4ef815df8373c` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:14.103254 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23627d4-b8ee-408c-85d7-047109e13105 admin admin] RBAC: Authorizing `identity:delete_project(project_id=17dd8d2e584a4cb69bf4ef815df8373c)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:14.105488 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23627d4-b8ee-408c-85d7-047109e13105 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:14.189747 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-f23627d4-b8ee-408c-85d7-047109e13105 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '17dd8d2e584a4cb69bf4ef815df8373c', 'request_id': 'req-f23627d4-b8ee-408c-85d7-047109e13105'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:14.196461 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-f23627d4-b8ee-408c-85d7-047109e13105 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '17dd8d2e584a4cb69bf4ef815df8373c', 'request_id': 'req-f23627d4-b8ee-408c-85d7-047109e13105'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:14.227078 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-f23627d4-b8ee-408c-85d7-047109e13105 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '17dd8d2e584a4cb69bf4ef815df8373c', 'request_id': 'req-f23627d4-b8ee-408c-85d7-047109e13105'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:14.250994 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-f23627d4-b8ee-408c-85d7-047109e13105 admin admin] The token cache is being invalidate because project 17dd8d2e584a4cb69bf4ef815df8373c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81759) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:41:14.251283 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-f23627d4-b8ee-408c-85d7-047109e13105 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:14.252514 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-f23627d4-b8ee-408c-85d7-047109e13105 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:14.257427 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 977/1957] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 17:41:13 2022] DELETE /identity/v3/projects/17dd8d2e584a4cb69bf4ef815df8373c => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:41:14.259476 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba834510-8420-48c4-89a8-bb1bcb29d9e2 None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:14.259966 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba834510-8420-48c4-89a8-bb1bcb29d9e2 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:14.260042 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba834510-8420-48c4-89a8-bb1bcb29d9e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:14.285923 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-ba834510-8420-48c4-89a8-bb1bcb29d9e2 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:41:14.286869 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-ba834510-8420-48c4-89a8-bb1bcb29d9e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:14.356929 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 981/1958] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:41:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:41:14.359274 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eca23bf-e553-4629-bc4c-d701029681e1 None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:14.359517 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eca23bf-e553-4629-bc4c-d701029681e1 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:14.359801 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eca23bf-e553-4629-bc4c-d701029681e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:14.384176 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-5eca23bf-e553-4629-bc4c-d701029681e1 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:41:14.385744 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-5eca23bf-e553-4629-bc4c-d701029681e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:14.400194 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 978/1959] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:41:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:41:14.401176 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba834510-8420-48c4-89a8-bb1bcb29d9e2 None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:14.411524 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba834510-8420-48c4-89a8-bb1bcb29d9e2 None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:14.425203 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9e48c1-dba9-442e-84d6-d8c0739c4db0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:14.425984 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9e48c1-dba9-442e-84d6-d8c0739c4db0 admin admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:14.426269 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9e48c1-dba9-442e-84d6-d8c0739c4db0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:14.426622 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9e48c1-dba9-442e-84d6-d8c0739c4db0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:14.427295 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9e48c1-dba9-442e-84d6-d8c0739c4db0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:41:14.427600 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9e48c1-dba9-442e-84d6-d8c0739c4db0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:14.432646 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9e48c1-dba9-442e-84d6-d8c0739c4db0 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:14.478995 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 982/1960] 10.222.0.45 () {58 vars in 1153 bytes} [Wed Aug 24 17:41:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:41:14.483327 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eca23bf-e553-4629-bc4c-d701029681e1 None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:14.494186 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eca23bf-e553-4629-bc4c-d701029681e1 None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:14.500447 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ce7f38-6ce2-444f-8c33-b277de089546 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:14.501166 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ce7f38-6ce2-444f-8c33-b277de089546 admin admin] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:14.501447 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ce7f38-6ce2-444f-8c33-b277de089546 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:14.502341 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ce7f38-6ce2-444f-8c33-b277de089546 admin admin] PATH_INFO: `/v3/projects` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:14.503995 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ce7f38-6ce2-444f-8c33-b277de089546 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:14.506492 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ce7f38-6ce2-444f-8c33-b277de089546 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:14.554409 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9e48c1-dba9-442e-84d6-d8c0739c4db0 admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:14.562392 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 979/1961] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 17:41:14 2022] POST /identity/v3/projects => generated 383 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:41:14.568973 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9e48c1-dba9-442e-84d6-d8c0739c4db0 admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:14.670313 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c43c88-27d9-406d-ae0d-659b0fa14e97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:14.671756 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c43c88-27d9-406d-ae0d-659b0fa14e97 admin admin] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:14.674270 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c43c88-27d9-406d-ae0d-659b0fa14e97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:14.675200 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c43c88-27d9-406d-ae0d-659b0fa14e97 admin admin] PATH_INFO: `/v3/users` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:14.677272 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c43c88-27d9-406d-ae0d-659b0fa14e97 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:14.681952 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c43c88-27d9-406d-ae0d-659b0fa14e97 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:14.722284 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 983/1962] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 17:41:14 2022] POST /identity/v3/users => generated 292 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:41:14.724504 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ce7f38-6ce2-444f-8c33-b277de089546 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:14.738412 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ce7f38-6ce2-444f-8c33-b277de089546 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:14.745786 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da0a1b6-6fef-477a-938b-193bff6b4572 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:14.746452 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da0a1b6-6fef-477a-938b-193bff6b4572 admin admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:14.746732 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da0a1b6-6fef-477a-938b-193bff6b4572 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:14.747073 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da0a1b6-6fef-477a-938b-193bff6b4572 admin admin] PATH_INFO: `/v3/roles` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:14.747780 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da0a1b6-6fef-477a-938b-193bff6b4572 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:14.749838 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da0a1b6-6fef-477a-938b-193bff6b4572 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:14.765281 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 980/1963] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:41:14 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:41:14.775325 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c43c88-27d9-406d-ae0d-659b0fa14e97 admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:14.786434 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c43c88-27d9-406d-ae0d-659b0fa14e97 admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:14.792765 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c4b8f0-8b50-4b63-a116-e51b1f6e6f04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:14.794131 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c4b8f0-8b50-4b63-a116-e51b1f6e6f04 admin admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:14.794131 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c4b8f0-8b50-4b63-a116-e51b1f6e6f04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:14.794131 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c4b8f0-8b50-4b63-a116-e51b1f6e6f04 admin admin] PATH_INFO: `/v3/roles` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:14.803984 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c4b8f0-8b50-4b63-a116-e51b1f6e6f04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:14.805685 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c4b8f0-8b50-4b63-a116-e51b1f6e6f04 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:14.821085 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 984/1964] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:41:14 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:41:14.823088 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da0a1b6-6fef-477a-938b-193bff6b4572 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:14.836189 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da0a1b6-6fef-477a-938b-193bff6b4572 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:14.841083 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6135261-616c-4713-a059-52236f9fc307 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:14.842232 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6135261-616c-4713-a059-52236f9fc307 admin admin] REQUEST_METHOD: `PUT` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:14.842232 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6135261-616c-4713-a059-52236f9fc307 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:14.842403 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6135261-616c-4713-a059-52236f9fc307 admin admin] PATH_INFO: `/v3/projects/19efca7af08e452f9200ff678db62c50/users/8ab1f8d8f90d4c0795c4818832e9a27b/roles/02476a5b82eb4b188c18a5d706a3ef68` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:14.867849 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6135261-616c-4713-a059-52236f9fc307 admin admin] RBAC: Authorizing `identity:create_grant(project_id=19efca7af08e452f9200ff678db62c50, user_id=8ab1f8d8f90d4c0795c4818832e9a27b, role_id=02476a5b82eb4b188c18a5d706a3ef68)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:14.879438 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6135261-616c-4713-a059-52236f9fc307 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:14.913136 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 981/1965] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 17:41:14 2022] PUT /identity/v3/projects/19efca7af08e452f9200ff678db62c50/users/8ab1f8d8f90d4c0795c4818832e9a27b/roles/02476a5b82eb4b188c18a5d706a3ef68 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:41:14.914801 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845d77f8-6702-463c-b869-96dfdfb2e07c None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:14.915114 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845d77f8-6702-463c-b869-96dfdfb2e07c None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:14.915392 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845d77f8-6702-463c-b869-96dfdfb2e07c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:14.958434 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-845d77f8-6702-463c-b869-96dfdfb2e07c None None] MFA Rules not processed for user `8ab1f8d8f90d4c0795c4818832e9a27b`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:41:14.960051 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-845d77f8-6702-463c-b869-96dfdfb2e07c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:14.968087 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:41:15.075703 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 985/1966] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:41:14 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:41:15.081892 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6135261-616c-4713-a059-52236f9fc307 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:15.179404 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6135261-616c-4713-a059-52236f9fc307 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:15.186488 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306ee243-698a-40d1-9736-1f9c19f7ac4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:15.186842 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306ee243-698a-40d1-9736-1f9c19f7ac4d admin admin] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:15.187093 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306ee243-698a-40d1-9736-1f9c19f7ac4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:15.187367 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306ee243-698a-40d1-9736-1f9c19f7ac4d admin admin] PATH_INFO: `/v3/projects` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:15.188003 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306ee243-698a-40d1-9736-1f9c19f7ac4d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:15.190024 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306ee243-698a-40d1-9736-1f9c19f7ac4d admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:15.211169 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:41:15.244824 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 982/1967] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 17:41:15 2022] POST /identity/v3/projects => generated 381 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:41:15.250087 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845d77f8-6702-463c-b869-96dfdfb2e07c None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:15.267937 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845d77f8-6702-463c-b869-96dfdfb2e07c None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:15.343104 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fd6e77-a7bf-41ab-8c90-493f42766024 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:15.343734 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fd6e77-a7bf-41ab-8c90-493f42766024 admin admin] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:15.344016 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fd6e77-a7bf-41ab-8c90-493f42766024 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:15.344289 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fd6e77-a7bf-41ab-8c90-493f42766024 admin admin] PATH_INFO: `/v3/users` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:15.344953 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fd6e77-a7bf-41ab-8c90-493f42766024 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:15.347491 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fd6e77-a7bf-41ab-8c90-493f42766024 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:15.370874 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 986/1968] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 17:41:15 2022] POST /identity/v3/users => generated 297 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:41:15.374787 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306ee243-698a-40d1-9736-1f9c19f7ac4d admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:15.389029 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306ee243-698a-40d1-9736-1f9c19f7ac4d admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:15.394792 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c124932-58d8-4b48-8794-56e4073dd174 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:15.395425 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c124932-58d8-4b48-8794-56e4073dd174 admin admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:15.395706 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c124932-58d8-4b48-8794-56e4073dd174 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:15.395981 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c124932-58d8-4b48-8794-56e4073dd174 admin admin] PATH_INFO: `/v3/roles` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:15.396675 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c124932-58d8-4b48-8794-56e4073dd174 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:15.398297 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c124932-58d8-4b48-8794-56e4073dd174 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:15.413537 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 983/1969] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:41:15 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:41:15.417539 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fd6e77-a7bf-41ab-8c90-493f42766024 admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:15.436775 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fd6e77-a7bf-41ab-8c90-493f42766024 admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:15.442693 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5154d913-3af7-495c-a32e-0f573528c9be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:15.443353 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5154d913-3af7-495c-a32e-0f573528c9be admin admin] REQUEST_METHOD: `PUT` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:15.443636 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5154d913-3af7-495c-a32e-0f573528c9be admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:15.443908 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5154d913-3af7-495c-a32e-0f573528c9be admin admin] PATH_INFO: `/v3/projects/705fef4d63374365ae82c1953e2c5562/users/26c77c910f0043fc85dd3570352c3db2/roles/e411ad9eb96e4e8d85b59b5b700ada7b` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:15.467011 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5154d913-3af7-495c-a32e-0f573528c9be admin admin] RBAC: Authorizing `identity:create_grant(project_id=705fef4d63374365ae82c1953e2c5562, user_id=26c77c910f0043fc85dd3570352c3db2, role_id=e411ad9eb96e4e8d85b59b5b700ada7b)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:15.470172 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5154d913-3af7-495c-a32e-0f573528c9be admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:15.488374 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 987/1970] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 17:41:15 2022] PUT /identity/v3/projects/705fef4d63374365ae82c1953e2c5562/users/26c77c910f0043fc85dd3570352c3db2/roles/e411ad9eb96e4e8d85b59b5b700ada7b => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:41:15.493166 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c124932-58d8-4b48-8794-56e4073dd174 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:15.507365 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c124932-58d8-4b48-8794-56e4073dd174 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:15.581293 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae17b416-6e04-47c8-a696-6c9f1588a33f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:15.583060 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae17b416-6e04-47c8-a696-6c9f1588a33f admin admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:15.583616 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae17b416-6e04-47c8-a696-6c9f1588a33f admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:15.584290 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae17b416-6e04-47c8-a696-6c9f1588a33f admin admin] PATH_INFO: `/v3/roles` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:15.585332 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae17b416-6e04-47c8-a696-6c9f1588a33f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:15.587801 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae17b416-6e04-47c8-a696-6c9f1588a33f admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:15.601824 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 984/1971] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:41:15 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:41:15.605607 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5154d913-3af7-495c-a32e-0f573528c9be admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:15.617725 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5154d913-3af7-495c-a32e-0f573528c9be admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:15.623989 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e967cb-a09d-496a-8045-d9f85590ae12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:15.624666 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e967cb-a09d-496a-8045-d9f85590ae12 admin admin] REQUEST_METHOD: `PUT` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:15.624952 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e967cb-a09d-496a-8045-d9f85590ae12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:15.625225 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e967cb-a09d-496a-8045-d9f85590ae12 admin admin] PATH_INFO: `/v3/projects/705fef4d63374365ae82c1953e2c5562/users/26c77c910f0043fc85dd3570352c3db2/roles/e411ad9eb96e4e8d85b59b5b700ada7b` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:15.628032 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e967cb-a09d-496a-8045-d9f85590ae12 admin admin] RBAC: Authorizing `identity:create_grant(project_id=705fef4d63374365ae82c1953e2c5562, user_id=26c77c910f0043fc85dd3570352c3db2, role_id=e411ad9eb96e4e8d85b59b5b700ada7b)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:15.630664 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e967cb-a09d-496a-8045-d9f85590ae12 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:15.644379 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 988/1972] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 17:41:15 2022] PUT /identity/v3/projects/705fef4d63374365ae82c1953e2c5562/users/26c77c910f0043fc85dd3570352c3db2/roles/e411ad9eb96e4e8d85b59b5b700ada7b => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:41:15.656544 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f7678c-13e6-41ff-9856-c84bfd7e10b1 None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:15.656544 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f7678c-13e6-41ff-9856-c84bfd7e10b1 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:15.656544 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f7678c-13e6-41ff-9856-c84bfd7e10b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:15.716694 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-a7f7678c-13e6-41ff-9856-c84bfd7e10b1 None None] MFA Rules not processed for user `26c77c910f0043fc85dd3570352c3db2`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:41:15.717933 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-a7f7678c-13e6-41ff-9856-c84bfd7e10b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:15.856999 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 985/1973] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:41:15 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:41:15.863342 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e967cb-a09d-496a-8045-d9f85590ae12 admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:15.865209 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-84e967cb-a09d-496a-8045-d9f85590ae12 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:15.946815 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e967cb-a09d-496a-8045-d9f85590ae12 admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:15.955670 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260d54c0-798f-419a-9474-401b27bbac96 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59be342541264942ad87f60ff4de6c8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:15.956838 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260d54c0-798f-419a-9474-401b27bbac96 service cinder] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:15.957191 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260d54c0-798f-419a-9474-401b27bbac96 service cinder] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:15.957537 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260d54c0-798f-419a-9474-401b27bbac96 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:15.979333 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260d54c0-798f-419a-9474-401b27bbac96 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:15.981112 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260d54c0-798f-419a-9474-401b27bbac96 service cinder] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:15.995754 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 989/1974] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:41:15 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:41:16.121973 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f7678c-13e6-41ff-9856-c84bfd7e10b1 None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:16.133020 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f7678c-13e6-41ff-9856-c84bfd7e10b1 None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:16.136953 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d069dec6-e7eb-48d0-a23c-fb818df5933a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59be342541264942ad87f60ff4de6c8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:16.137910 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d069dec6-e7eb-48d0-a23c-fb818df5933a service cinder] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:16.138194 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d069dec6-e7eb-48d0-a23c-fb818df5933a service cinder] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:16.138469 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d069dec6-e7eb-48d0-a23c-fb818df5933a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:16.208704 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d069dec6-e7eb-48d0-a23c-fb818df5933a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:16.210172 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d069dec6-e7eb-48d0-a23c-fb818df5933a service cinder] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:16.220069 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 986/1975] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:41:16 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:41:16.700384 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260d54c0-798f-419a-9474-401b27bbac96 service cinder] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:16.704607 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:41:16.794034 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260d54c0-798f-419a-9474-401b27bbac96 service cinder] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:16.800285 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd642336-2048-417c-845b-d8ab1a17db13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:16.801222 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd642336-2048-417c-845b-d8ab1a17db13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:16.801788 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd642336-2048-417c-845b-d8ab1a17db13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:16.802239 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd642336-2048-417c-845b-d8ab1a17db13 admin admin] PATH_INFO: `/v3/users/8ab1f8d8f90d4c0795c4818832e9a27b` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:16.803670 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd642336-2048-417c-845b-d8ab1a17db13 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8ab1f8d8f90d4c0795c4818832e9a27b)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:16.806980 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd642336-2048-417c-845b-d8ab1a17db13 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:16.907555 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-dd642336-2048-417c-845b-d8ab1a17db13 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8ab1f8d8f90d4c0795c4818832e9a27b'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:16.908007 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-dd642336-2048-417c-845b-d8ab1a17db13 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8ab1f8d8f90d4c0795c4818832e9a27b'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:16.919946 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-dd642336-2048-417c-845b-d8ab1a17db13 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8ab1f8d8f90d4c0795c4818832e9a27b'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:16.920660 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-dd642336-2048-417c-845b-d8ab1a17db13 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8ab1f8d8f90d4c0795c4818832e9a27b'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:16.937667 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-dd642336-2048-417c-845b-d8ab1a17db13 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8ab1f8d8f90d4c0795c4818832e9a27b'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:16.965333 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 990/1976] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 17:41:16 2022] DELETE /identity/v3/users/8ab1f8d8f90d4c0795c4818832e9a27b => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:41:16.970036 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee92ffe-4903-48fb-a5ea-9c3e8614f764 None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:16.971393 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee92ffe-4903-48fb-a5ea-9c3e8614f764 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:16.971821 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee92ffe-4903-48fb-a5ea-9c3e8614f764 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:16.985975 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:41:17.027142 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4553fc70-0503-466f-a0f4-edd692595ed2 None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:17.027426 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4553fc70-0503-466f-a0f4-edd692595ed2 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:17.027712 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4553fc70-0503-466f-a0f4-edd692595ed2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:17.028708 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-2ee92ffe-4903-48fb-a5ea-9c3e8614f764 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:41:17.035555 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-2ee92ffe-4903-48fb-a5ea-9c3e8614f764 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:17.104060 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-4553fc70-0503-466f-a0f4-edd692595ed2 None None] MFA Rules not processed for user `af6710bfe62d4ba29dc48e2f7aba9ec5`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:41:17.105315 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-4553fc70-0503-466f-a0f4-edd692595ed2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:17.161041 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 987/1977] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:41:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:41:17.173539 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee92ffe-4903-48fb-a5ea-9c3e8614f764 None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:17.177281 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-2ee92ffe-4903-48fb-a5ea-9c3e8614f764 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:17.277787 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 991/1978] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:41:17 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:41:17.282180 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4553fc70-0503-466f-a0f4-edd692595ed2 None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:17.285624 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-4553fc70-0503-466f-a0f4-edd692595ed2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:17.294919 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee92ffe-4903-48fb-a5ea-9c3e8614f764 None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:17.305569 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fa3e35-36e0-4eb1-8241-6b3e1a25c9a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43ad98a250a144099ccd18378aedd4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:17.306759 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fa3e35-36e0-4eb1-8241-6b3e1a25c9a5 service neutron] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:17.307133 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fa3e35-36e0-4eb1-8241-6b3e1a25c9a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:17.307836 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fa3e35-36e0-4eb1-8241-6b3e1a25c9a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:17.311884 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4553fc70-0503-466f-a0f4-edd692595ed2 None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:17.324887 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f96fdc9-d556-4633-99f4-4ca522d220cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:17.325639 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f96fdc9-d556-4633-99f4-4ca522d220cd admin admin] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:17.325915 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f96fdc9-d556-4633-99f4-4ca522d220cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:17.326236 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f96fdc9-d556-4633-99f4-4ca522d220cd admin admin] PATH_INFO: `/v3/projects` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:17.326861 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fa3e35-36e0-4eb1-8241-6b3e1a25c9a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:17.326998 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f96fdc9-d556-4633-99f4-4ca522d220cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:17.328739 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fa3e35-36e0-4eb1-8241-6b3e1a25c9a5 service neutron] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:17.329028 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f96fdc9-d556-4633-99f4-4ca522d220cd admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:17.345374 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 988/1979] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:41:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:41:17.349524 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:41:17.432366 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 992/1980] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 17:41:17 2022] POST /identity/v3/projects => generated 379 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:41:17.436461 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fa3e35-36e0-4eb1-8241-6b3e1a25c9a5 service neutron] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:17.451971 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fa3e35-36e0-4eb1-8241-6b3e1a25c9a5 service neutron] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:17.583325 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60177014-62a9-45d7-b7ec-16692b440f80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:17.584331 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60177014-62a9-45d7-b7ec-16692b440f80 admin admin] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:17.584798 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60177014-62a9-45d7-b7ec-16692b440f80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:17.585309 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60177014-62a9-45d7-b7ec-16692b440f80 admin admin] PATH_INFO: `/v3/users` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:17.586134 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60177014-62a9-45d7-b7ec-16692b440f80 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:17.590893 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60177014-62a9-45d7-b7ec-16692b440f80 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:17.645499 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 989/1981] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 17:41:17 2022] POST /identity/v3/users => generated 296 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:41:17.651681 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f96fdc9-d556-4633-99f4-4ca522d220cd admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:17.667356 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f96fdc9-d556-4633-99f4-4ca522d220cd admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:17.680319 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56d8a29-7ae5-4ca7-9e71-6ac6fe8769d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:17.680794 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56d8a29-7ae5-4ca7-9e71-6ac6fe8769d9 admin admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:17.681073 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56d8a29-7ae5-4ca7-9e71-6ac6fe8769d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:17.681347 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56d8a29-7ae5-4ca7-9e71-6ac6fe8769d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:17.682156 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56d8a29-7ae5-4ca7-9e71-6ac6fe8769d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:17.684064 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56d8a29-7ae5-4ca7-9e71-6ac6fe8769d9 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:17.721366 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 993/1982] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:41:17 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:41:17.725847 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60177014-62a9-45d7-b7ec-16692b440f80 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:17.741995 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60177014-62a9-45d7-b7ec-16692b440f80 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:17.751242 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0758dbe7-5fdf-4b3a-b231-a01992fcfba9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:17.752980 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0758dbe7-5fdf-4b3a-b231-a01992fcfba9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:17.753411 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0758dbe7-5fdf-4b3a-b231-a01992fcfba9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:17.753826 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0758dbe7-5fdf-4b3a-b231-a01992fcfba9 admin admin] PATH_INFO: `/v3/projects/a8040afb734748a4ac1698d26f7321fa/users/3c60d052608a4dce99944f07912a36f2/roles/e411ad9eb96e4e8d85b59b5b700ada7b` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:17.786389 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0758dbe7-5fdf-4b3a-b231-a01992fcfba9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a8040afb734748a4ac1698d26f7321fa, user_id=3c60d052608a4dce99944f07912a36f2, role_id=e411ad9eb96e4e8d85b59b5b700ada7b)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:17.806094 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0758dbe7-5fdf-4b3a-b231-a01992fcfba9 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:17.835783 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 990/1983] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 17:41:17 2022] PUT /identity/v3/projects/a8040afb734748a4ac1698d26f7321fa/users/3c60d052608a4dce99944f07912a36f2/roles/e411ad9eb96e4e8d85b59b5b700ada7b => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:41:17.839805 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56d8a29-7ae5-4ca7-9e71-6ac6fe8769d9 admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:17.857294 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56d8a29-7ae5-4ca7-9e71-6ac6fe8769d9 admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:17.864999 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0758dbe7-5fdf-4b3a-b231-a01992fcfba9 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:17.866892 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-0758dbe7-5fdf-4b3a-b231-a01992fcfba9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:17.889418 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:41:17.978842 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0758dbe7-5fdf-4b3a-b231-a01992fcfba9 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:17.984802 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630b8cac-df79-46ef-997e-d60fb8c32b1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:17.985357 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630b8cac-df79-46ef-997e-d60fb8c32b1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:17.986556 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630b8cac-df79-46ef-997e-d60fb8c32b1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:17.986556 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630b8cac-df79-46ef-997e-d60fb8c32b1b admin admin] PATH_INFO: `/v3/projects/19efca7af08e452f9200ff678db62c50` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:17.987822 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630b8cac-df79-46ef-997e-d60fb8c32b1b admin admin] RBAC: Authorizing `identity:delete_project(project_id=19efca7af08e452f9200ff678db62c50)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:17.990886 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630b8cac-df79-46ef-997e-d60fb8c32b1b admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:18.055692 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f32720-8512-4052-acda-72733dfecf91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:18.056755 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f32720-8512-4052-acda-72733dfecf91 admin admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:18.057240 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f32720-8512-4052-acda-72733dfecf91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:18.057915 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f32720-8512-4052-acda-72733dfecf91 admin admin] PATH_INFO: `/v3/roles` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:18.059205 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f32720-8512-4052-acda-72733dfecf91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:18.061050 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f32720-8512-4052-acda-72733dfecf91 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:18.063370 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-630b8cac-df79-46ef-997e-d60fb8c32b1b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '19efca7af08e452f9200ff678db62c50', 'request_id': 'req-630b8cac-df79-46ef-997e-d60fb8c32b1b'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:18.072418 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 994/1984] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:41:17 2022] GET /identity/v3/roles => generated 1420 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:41:18.073262 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-630b8cac-df79-46ef-997e-d60fb8c32b1b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '19efca7af08e452f9200ff678db62c50', 'request_id': 'req-630b8cac-df79-46ef-997e-d60fb8c32b1b'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:18.076894 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f32720-8512-4052-acda-72733dfecf91 admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:18.082075 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-630b8cac-df79-46ef-997e-d60fb8c32b1b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '19efca7af08e452f9200ff678db62c50', 'request_id': 'req-630b8cac-df79-46ef-997e-d60fb8c32b1b'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:18.088766 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f32720-8512-4052-acda-72733dfecf91 admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:18.103319 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-630b8cac-df79-46ef-997e-d60fb8c32b1b admin admin] The token cache is being invalidate because project 19efca7af08e452f9200ff678db62c50 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81759) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:41:18.104498 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-630b8cac-df79-46ef-997e-d60fb8c32b1b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:18.105366 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-630b8cac-df79-46ef-997e-d60fb8c32b1b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:18.107175 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 991/1985] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 17:41:17 2022] DELETE /identity/v3/projects/19efca7af08e452f9200ff678db62c50 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:41:18.112846 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630b8cac-df79-46ef-997e-d60fb8c32b1b admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:18.114811 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-630b8cac-df79-46ef-997e-d60fb8c32b1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:18.184199 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdb3b48-c6c0-42d1-8ab1-6af51fb79f76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:18.185044 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdb3b48-c6c0-42d1-8ab1-6af51fb79f76 admin admin] REQUEST_METHOD: `PUT` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:18.185337 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdb3b48-c6c0-42d1-8ab1-6af51fb79f76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:18.185588 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdb3b48-c6c0-42d1-8ab1-6af51fb79f76 admin admin] PATH_INFO: `/v3/projects/a8040afb734748a4ac1698d26f7321fa/users/3c60d052608a4dce99944f07912a36f2/roles/e411ad9eb96e4e8d85b59b5b700ada7b` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:18.187921 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdb3b48-c6c0-42d1-8ab1-6af51fb79f76 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a8040afb734748a4ac1698d26f7321fa, user_id=3c60d052608a4dce99944f07912a36f2, role_id=e411ad9eb96e4e8d85b59b5b700ada7b)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:18.191078 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdb3b48-c6c0-42d1-8ab1-6af51fb79f76 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:18.207820 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 995/1986] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 17:41:18 2022] PUT /identity/v3/projects/a8040afb734748a4ac1698d26f7321fa/users/3c60d052608a4dce99944f07912a36f2/roles/e411ad9eb96e4e8d85b59b5b700ada7b => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:41:18.231998 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630b8cac-df79-46ef-997e-d60fb8c32b1b admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:18.331725 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:41:18.358234 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e5cc10-d935-4e5a-96a3-96f44d6657c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:18.359249 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e5cc10-d935-4e5a-96a3-96f44d6657c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:18.360418 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e5cc10-d935-4e5a-96a3-96f44d6657c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:18.361427 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e5cc10-d935-4e5a-96a3-96f44d6657c3 admin admin] PATH_INFO: `/v3/users/26c77c910f0043fc85dd3570352c3db2` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:18.367746 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e5cc10-d935-4e5a-96a3-96f44d6657c3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=26c77c910f0043fc85dd3570352c3db2)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:18.371811 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e5cc10-d935-4e5a-96a3-96f44d6657c3 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:18.484426 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-06e5cc10-d935-4e5a-96a3-96f44d6657c3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '26c77c910f0043fc85dd3570352c3db2'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:18.485868 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-06e5cc10-d935-4e5a-96a3-96f44d6657c3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '26c77c910f0043fc85dd3570352c3db2'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:18.496620 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-06e5cc10-d935-4e5a-96a3-96f44d6657c3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '26c77c910f0043fc85dd3570352c3db2'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:18.498064 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-06e5cc10-d935-4e5a-96a3-96f44d6657c3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '26c77c910f0043fc85dd3570352c3db2'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:18.513290 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-06e5cc10-d935-4e5a-96a3-96f44d6657c3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '26c77c910f0043fc85dd3570352c3db2'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:18.549965 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 992/1987] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 17:41:18 2022] DELETE /identity/v3/users/26c77c910f0043fc85dd3570352c3db2 => generated 0 bytes in 439 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:41:19.186660 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdb3b48-c6c0-42d1-8ab1-6af51fb79f76 admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:19.191798 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-6cdb3b48-c6c0-42d1-8ab1-6af51fb79f76 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:19.211278 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdb3b48-c6c0-42d1-8ab1-6af51fb79f76 admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:19.342437 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c122b861-20c0-4349-aed9-3155f1273c12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:19.343207 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c122b861-20c0-4349-aed9-3155f1273c12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:19.343627 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c122b861-20c0-4349-aed9-3155f1273c12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:19.344041 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c122b861-20c0-4349-aed9-3155f1273c12 admin admin] PATH_INFO: `/v3/projects/705fef4d63374365ae82c1953e2c5562` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:19.345989 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c122b861-20c0-4349-aed9-3155f1273c12 admin admin] RBAC: Authorizing `identity:delete_project(project_id=705fef4d63374365ae82c1953e2c5562)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:19.348595 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c122b861-20c0-4349-aed9-3155f1273c12 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:19.372316 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:41:19.447470 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-c122b861-20c0-4349-aed9-3155f1273c12 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '705fef4d63374365ae82c1953e2c5562', 'request_id': 'req-c122b861-20c0-4349-aed9-3155f1273c12'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:19.467131 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-c122b861-20c0-4349-aed9-3155f1273c12 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '705fef4d63374365ae82c1953e2c5562', 'request_id': 'req-c122b861-20c0-4349-aed9-3155f1273c12'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:19.474794 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-c122b861-20c0-4349-aed9-3155f1273c12 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '705fef4d63374365ae82c1953e2c5562', 'request_id': 'req-c122b861-20c0-4349-aed9-3155f1273c12'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:19.499425 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-c122b861-20c0-4349-aed9-3155f1273c12 admin admin] The token cache is being invalidate because project 705fef4d63374365ae82c1953e2c5562 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81758) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:41:19.500106 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-c122b861-20c0-4349-aed9-3155f1273c12 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:19.501218 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-c122b861-20c0-4349-aed9-3155f1273c12 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:19.504540 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 996/1988] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 17:41:19 2022] DELETE /identity/v3/projects/705fef4d63374365ae82c1953e2c5562 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:41:19.509295 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c989bb-1b78-4196-bd36-c10904af921c None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:19.509788 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c989bb-1b78-4196-bd36-c10904af921c None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:19.509864 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c989bb-1b78-4196-bd36-c10904af921c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:19.543373 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-c7c989bb-1b78-4196-bd36-c10904af921c None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:41:19.544606 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-c7c989bb-1b78-4196-bd36-c10904af921c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:19.677910 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18a91b4-b290-49d6-8382-d02ffaf1c8bd None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:19.678109 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 993/1989] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:41:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:41:19.678192 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18a91b4-b290-49d6-8382-d02ffaf1c8bd None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:19.678451 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18a91b4-b290-49d6-8382-d02ffaf1c8bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:19.720131 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-f18a91b4-b290-49d6-8382-d02ffaf1c8bd None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:41:19.725459 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-f18a91b4-b290-49d6-8382-d02ffaf1c8bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:19.749630 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 997/1990] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:41:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:41:19.756482 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c989bb-1b78-4196-bd36-c10904af921c None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:19.794810 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c989bb-1b78-4196-bd36-c10904af921c None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:19.824433 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498b94e9-13dc-4f6e-a2e8-5b8213c12731 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:19.827593 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498b94e9-13dc-4f6e-a2e8-5b8213c12731 admin admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:19.827894 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498b94e9-13dc-4f6e-a2e8-5b8213c12731 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:19.831707 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498b94e9-13dc-4f6e-a2e8-5b8213c12731 admin admin] PATH_INFO: `/v3/domains` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:19.834247 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498b94e9-13dc-4f6e-a2e8-5b8213c12731 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:41:19.834567 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498b94e9-13dc-4f6e-a2e8-5b8213c12731 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:19.838190 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498b94e9-13dc-4f6e-a2e8-5b8213c12731 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:19.874899 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 994/1991] 10.222.0.45 () {58 vars in 1153 bytes} [Wed Aug 24 17:41:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:41:19.880230 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18a91b4-b290-49d6-8382-d02ffaf1c8bd None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:19.893382 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18a91b4-b290-49d6-8382-d02ffaf1c8bd None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:19.906418 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1621d0c2-f0f8-437f-abf4-ee1a9a19f42e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:19.907047 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1621d0c2-f0f8-437f-abf4-ee1a9a19f42e admin admin] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:19.907327 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1621d0c2-f0f8-437f-abf4-ee1a9a19f42e admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:19.907926 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1621d0c2-f0f8-437f-abf4-ee1a9a19f42e admin admin] PATH_INFO: `/v3/projects` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:19.908276 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1621d0c2-f0f8-437f-abf4-ee1a9a19f42e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:19.910879 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1621d0c2-f0f8-437f-abf4-ee1a9a19f42e admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:19.963285 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 998/1992] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 17:41:19 2022] POST /identity/v3/projects => generated 377 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:41:19.963579 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498b94e9-13dc-4f6e-a2e8-5b8213c12731 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:19.977259 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498b94e9-13dc-4f6e-a2e8-5b8213c12731 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:20.056801 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:41:20.069727 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be08b898-2f3a-4603-acbe-45def00792ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:20.070719 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be08b898-2f3a-4603-acbe-45def00792ca admin admin] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:20.071139 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be08b898-2f3a-4603-acbe-45def00792ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:20.071550 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be08b898-2f3a-4603-acbe-45def00792ca admin admin] PATH_INFO: `/v3/users` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:20.072336 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be08b898-2f3a-4603-acbe-45def00792ca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:20.074828 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be08b898-2f3a-4603-acbe-45def00792ca admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:20.106132 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 995/1993] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 17:41:19 2022] POST /identity/v3/users => generated 289 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:41:20.111937 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1621d0c2-f0f8-437f-abf4-ee1a9a19f42e admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:20.127781 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1621d0c2-f0f8-437f-abf4-ee1a9a19f42e admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:20.133824 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e6cf91-64b3-408e-a22d-661188d0c008 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:20.134457 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e6cf91-64b3-408e-a22d-661188d0c008 admin admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:20.134746 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e6cf91-64b3-408e-a22d-661188d0c008 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:20.135027 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e6cf91-64b3-408e-a22d-661188d0c008 admin admin] PATH_INFO: `/v3/roles` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:20.135677 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e6cf91-64b3-408e-a22d-661188d0c008 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:20.137587 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e6cf91-64b3-408e-a22d-661188d0c008 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:20.154684 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 999/1994] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:41:20 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:41:20.155701 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be08b898-2f3a-4603-acbe-45def00792ca admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:20.178504 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be08b898-2f3a-4603-acbe-45def00792ca admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:20.188686 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905fb21d-60a3-475d-a64f-8ece065daad7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:20.189401 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905fb21d-60a3-475d-a64f-8ece065daad7 admin admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:20.189684 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905fb21d-60a3-475d-a64f-8ece065daad7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:20.189957 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905fb21d-60a3-475d-a64f-8ece065daad7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:20.190612 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905fb21d-60a3-475d-a64f-8ece065daad7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:20.192271 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905fb21d-60a3-475d-a64f-8ece065daad7 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:20.205490 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 996/1995] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:41:20 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:41:20.209378 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e6cf91-64b3-408e-a22d-661188d0c008 admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:20.222363 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e6cf91-64b3-408e-a22d-661188d0c008 admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:20.228484 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ae5453-9d37-456d-a133-f6468ff9a504 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:20.229002 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ae5453-9d37-456d-a133-f6468ff9a504 admin admin] REQUEST_METHOD: `PUT` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:20.230234 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ae5453-9d37-456d-a133-f6468ff9a504 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:20.230234 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ae5453-9d37-456d-a133-f6468ff9a504 admin admin] PATH_INFO: `/v3/projects/3d8c7e71553e4b53b71e9043cec31d4b/users/8460d762680649339dc1d5398a9094ee/roles/02476a5b82eb4b188c18a5d706a3ef68` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:20.252033 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ae5453-9d37-456d-a133-f6468ff9a504 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d8c7e71553e4b53b71e9043cec31d4b, user_id=8460d762680649339dc1d5398a9094ee, role_id=02476a5b82eb4b188c18a5d706a3ef68)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:20.265346 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ae5453-9d37-456d-a133-f6468ff9a504 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:20.280596 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1000/1996] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 17:41:20 2022] PUT /identity/v3/projects/3d8c7e71553e4b53b71e9043cec31d4b/users/8460d762680649339dc1d5398a9094ee/roles/02476a5b82eb4b188c18a5d706a3ef68 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:41:20.292430 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14fe41b-bc7e-4fb9-af6d-f8f2437839a0 None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:20.292750 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14fe41b-bc7e-4fb9-af6d-f8f2437839a0 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:20.293141 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14fe41b-bc7e-4fb9-af6d-f8f2437839a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:20.338448 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-b14fe41b-bc7e-4fb9-af6d-f8f2437839a0 None None] MFA Rules not processed for user `8460d762680649339dc1d5398a9094ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:41:20.339828 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-b14fe41b-bc7e-4fb9-af6d-f8f2437839a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:20.560103 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 997/1997] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:41:20 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:41:20.568918 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ae5453-9d37-456d-a133-f6468ff9a504 admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:20.573081 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-21ae5453-9d37-456d-a133-f6468ff9a504 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:20.729820 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ae5453-9d37-456d-a133-f6468ff9a504 admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:20.734616 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3ccac5-1489-48c7-8fb3-700deb06473e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59be342541264942ad87f60ff4de6c8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:20.735374 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3ccac5-1489-48c7-8fb3-700deb06473e service cinder] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:20.735785 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3ccac5-1489-48c7-8fb3-700deb06473e service cinder] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:20.736184 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3ccac5-1489-48c7-8fb3-700deb06473e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:20.749156 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3ccac5-1489-48c7-8fb3-700deb06473e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:20.750877 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3ccac5-1489-48c7-8fb3-700deb06473e service cinder] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:20.762066 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1001/1998] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:41:20 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:41:20.850670 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14fe41b-bc7e-4fb9-af6d-f8f2437839a0 None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:20.852248 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-b14fe41b-bc7e-4fb9-af6d-f8f2437839a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:21.035257 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14fe41b-bc7e-4fb9-af6d-f8f2437839a0 None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:21.041590 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6e32ea-bc42-4e1e-bc22-c9ce10b8b3d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:21.042504 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6e32ea-bc42-4e1e-bc22-c9ce10b8b3d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:21.042963 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6e32ea-bc42-4e1e-bc22-c9ce10b8b3d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:21.043491 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6e32ea-bc42-4e1e-bc22-c9ce10b8b3d3 admin admin] PATH_INFO: `/v3/users/cd19c2803225410988976133b98bcf30` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:21.044897 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6e32ea-bc42-4e1e-bc22-c9ce10b8b3d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cd19c2803225410988976133b98bcf30)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:21.047280 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6e32ea-bc42-4e1e-bc22-c9ce10b8b3d3 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:21.195138 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-1b6e32ea-bc42-4e1e-bc22-c9ce10b8b3d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cd19c2803225410988976133b98bcf30'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:21.197003 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-1b6e32ea-bc42-4e1e-bc22-c9ce10b8b3d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cd19c2803225410988976133b98bcf30'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:21.222251 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-1b6e32ea-bc42-4e1e-bc22-c9ce10b8b3d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cd19c2803225410988976133b98bcf30'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:21.223007 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-1b6e32ea-bc42-4e1e-bc22-c9ce10b8b3d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cd19c2803225410988976133b98bcf30'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:21.241139 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-1b6e32ea-bc42-4e1e-bc22-c9ce10b8b3d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cd19c2803225410988976133b98bcf30'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:21.245091 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:41:21.278056 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 998/1999] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 17:41:20 2022] DELETE /identity/v3/users/cd19c2803225410988976133b98bcf30 => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:41:21.282580 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1338af-c9e6-4fe2-9789-2133ca3696bb None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:21.283132 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1338af-c9e6-4fe2-9789-2133ca3696bb None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:21.283218 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1338af-c9e6-4fe2-9789-2133ca3696bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:21.325088 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-7f1338af-c9e6-4fe2-9789-2133ca3696bb None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:41:21.326338 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6e32ea-bc42-4e1e-bc22-c9ce10b8b3d3 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:21.328074 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-7f1338af-c9e6-4fe2-9789-2133ca3696bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:21.329325 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-1b6e32ea-bc42-4e1e-bc22-c9ce10b8b3d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:21.410479 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:41:21.459958 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6e32ea-bc42-4e1e-bc22-c9ce10b8b3d3 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:21.471280 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0c3b8a-a391-4d2b-8427-ac7955e26b61 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59be342541264942ad87f60ff4de6c8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:21.471917 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0c3b8a-a391-4d2b-8427-ac7955e26b61 service cinder] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:21.472201 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0c3b8a-a391-4d2b-8427-ac7955e26b61 service cinder] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:21.472602 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0c3b8a-a391-4d2b-8427-ac7955e26b61 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:21.472971 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1002/2000] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:41:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:41:21.478819 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-bf0c3b8a-a391-4d2b-8427-ac7955e26b61 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:21.529699 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1338af-c9e6-4fe2-9789-2133ca3696bb None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:21.534493 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-7f1338af-c9e6-4fe2-9789-2133ca3696bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:21.691275 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0c3b8a-a391-4d2b-8427-ac7955e26b61 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:21.693557 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0c3b8a-a391-4d2b-8427-ac7955e26b61 service cinder] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:21.696511 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1338af-c9e6-4fe2-9789-2133ca3696bb None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:21.707196 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8443e5f-c313-4b03-bd78-7c74f4894780 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43ad98a250a144099ccd18378aedd4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:21.708143 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8443e5f-c313-4b03-bd78-7c74f4894780 service neutron] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:21.708640 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8443e5f-c313-4b03-bd78-7c74f4894780 service neutron] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:21.711371 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8443e5f-c313-4b03-bd78-7c74f4894780 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:21.712424 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 999/2001] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:41:21 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:41:21.727964 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8443e5f-c313-4b03-bd78-7c74f4894780 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:21.729893 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8443e5f-c313-4b03-bd78-7c74f4894780 service neutron] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:21.744359 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1003/2002] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:41:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:41:23.517963 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0c3b8a-a391-4d2b-8427-ac7955e26b61 service cinder] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:23.519464 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-bf0c3b8a-a391-4d2b-8427-ac7955e26b61 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:23.545661 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0c3b8a-a391-4d2b-8427-ac7955e26b61 service cinder] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:23.557249 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13859fe-eff8-4a42-80eb-cd00ac89a826 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:23.558130 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13859fe-eff8-4a42-80eb-cd00ac89a826 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:23.559571 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13859fe-eff8-4a42-80eb-cd00ac89a826 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:23.560023 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13859fe-eff8-4a42-80eb-cd00ac89a826 admin admin] PATH_INFO: `/v3/projects/b035c069f2624ff2851fee7414f8c390` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:23.568488 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13859fe-eff8-4a42-80eb-cd00ac89a826 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b035c069f2624ff2851fee7414f8c390)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:23.570468 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13859fe-eff8-4a42-80eb-cd00ac89a826 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:23.701147 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-f13859fe-eff8-4a42-80eb-cd00ac89a826 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b035c069f2624ff2851fee7414f8c390', 'request_id': 'req-f13859fe-eff8-4a42-80eb-cd00ac89a826'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:23.717355 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-f13859fe-eff8-4a42-80eb-cd00ac89a826 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b035c069f2624ff2851fee7414f8c390', 'request_id': 'req-f13859fe-eff8-4a42-80eb-cd00ac89a826'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:23.731303 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-f13859fe-eff8-4a42-80eb-cd00ac89a826 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b035c069f2624ff2851fee7414f8c390', 'request_id': 'req-f13859fe-eff8-4a42-80eb-cd00ac89a826'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:23.763577 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-f13859fe-eff8-4a42-80eb-cd00ac89a826 admin admin] The token cache is being invalidate because project b035c069f2624ff2851fee7414f8c390 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81759) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:41:23.765288 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-f13859fe-eff8-4a42-80eb-cd00ac89a826 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:23.766074 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-f13859fe-eff8-4a42-80eb-cd00ac89a826 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:23.773112 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1000/2003] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 17:41:23 2022] DELETE /identity/v3/projects/b035c069f2624ff2851fee7414f8c390 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:41:23.774359 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8443e5f-c313-4b03-bd78-7c74f4894780 service neutron] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:23.793439 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8443e5f-c313-4b03-bd78-7c74f4894780 service neutron] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:23.946896 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff280b0-5d82-41a4-b1ef-17f0db6008a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:23.947324 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff280b0-5d82-41a4-b1ef-17f0db6008a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:23.947784 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff280b0-5d82-41a4-b1ef-17f0db6008a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:23.948228 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff280b0-5d82-41a4-b1ef-17f0db6008a9 admin admin] PATH_INFO: `/v3/users/7bf0ab87f39d4ae0aa21f87faff1ed6c` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:23.953867 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff280b0-5d82-41a4-b1ef-17f0db6008a9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7bf0ab87f39d4ae0aa21f87faff1ed6c)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:23.956388 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff280b0-5d82-41a4-b1ef-17f0db6008a9 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:24.154029 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-5ff280b0-5d82-41a4-b1ef-17f0db6008a9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7bf0ab87f39d4ae0aa21f87faff1ed6c'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:24.157731 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-5ff280b0-5d82-41a4-b1ef-17f0db6008a9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7bf0ab87f39d4ae0aa21f87faff1ed6c'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:24.180257 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-5ff280b0-5d82-41a4-b1ef-17f0db6008a9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7bf0ab87f39d4ae0aa21f87faff1ed6c'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:24.182400 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-5ff280b0-5d82-41a4-b1ef-17f0db6008a9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7bf0ab87f39d4ae0aa21f87faff1ed6c'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:24.227313 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-5ff280b0-5d82-41a4-b1ef-17f0db6008a9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7bf0ab87f39d4ae0aa21f87faff1ed6c'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:24.251017 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:41:24.263515 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1004/2004] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 17:41:23 2022] DELETE /identity/v3/users/7bf0ab87f39d4ae0aa21f87faff1ed6c => generated 0 bytes in 484 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:41:25.094650 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13859fe-eff8-4a42-80eb-cd00ac89a826 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:25.100291 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-f13859fe-eff8-4a42-80eb-cd00ac89a826 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:25.134705 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13859fe-eff8-4a42-80eb-cd00ac89a826 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:25.185063 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:41:25.263542 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90569e94-6f7c-4860-8efc-754683545ad0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:25.264645 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90569e94-6f7c-4860-8efc-754683545ad0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:25.264932 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90569e94-6f7c-4860-8efc-754683545ad0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:25.265210 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90569e94-6f7c-4860-8efc-754683545ad0 admin admin] PATH_INFO: `/v3/projects/42a6b11e6b0e4116bee451839753316f` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:25.266538 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90569e94-6f7c-4860-8efc-754683545ad0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=42a6b11e6b0e4116bee451839753316f)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:25.269242 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90569e94-6f7c-4860-8efc-754683545ad0 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:25.347285 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-90569e94-6f7c-4860-8efc-754683545ad0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '42a6b11e6b0e4116bee451839753316f', 'request_id': 'req-90569e94-6f7c-4860-8efc-754683545ad0'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:25.358249 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-90569e94-6f7c-4860-8efc-754683545ad0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '42a6b11e6b0e4116bee451839753316f', 'request_id': 'req-90569e94-6f7c-4860-8efc-754683545ad0'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:25.368933 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-90569e94-6f7c-4860-8efc-754683545ad0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '42a6b11e6b0e4116bee451839753316f', 'request_id': 'req-90569e94-6f7c-4860-8efc-754683545ad0'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:25.390968 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-90569e94-6f7c-4860-8efc-754683545ad0 admin admin] The token cache is being invalidate because project 42a6b11e6b0e4116bee451839753316f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81759) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:41:25.391277 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-90569e94-6f7c-4860-8efc-754683545ad0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:25.392076 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-90569e94-6f7c-4860-8efc-754683545ad0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:25.393741 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1001/2005] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 17:41:25 2022] DELETE /identity/v3/projects/42a6b11e6b0e4116bee451839753316f => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:41:25.398707 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f117ceb-3f6d-463a-8682-e0d69cf5c5e0 None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:25.399191 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f117ceb-3f6d-463a-8682-e0d69cf5c5e0 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:25.399266 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f117ceb-3f6d-463a-8682-e0d69cf5c5e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:25.469846 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-8f117ceb-3f6d-463a-8682-e0d69cf5c5e0 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:41:25.473937 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-8f117ceb-3f6d-463a-8682-e0d69cf5c5e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:25.655930 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1005/2006] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:41:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:41:25.664108 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fdc54b-c93c-486f-b602-1e4bd8fb19f9 None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:25.664460 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fdc54b-c93c-486f-b602-1e4bd8fb19f9 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:25.665201 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fdc54b-c93c-486f-b602-1e4bd8fb19f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:25.696677 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-c0fdc54b-c93c-486f-b602-1e4bd8fb19f9 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:41:25.698250 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-c0fdc54b-c93c-486f-b602-1e4bd8fb19f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:25.730352 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1002/2007] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:41:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:41:25.730993 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f117ceb-3f6d-463a-8682-e0d69cf5c5e0 None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:25.769467 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f117ceb-3f6d-463a-8682-e0d69cf5c5e0 None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:25.777120 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4b8854-feb3-40b0-8028-0cf253159ac9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:25.778255 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4b8854-feb3-40b0-8028-0cf253159ac9 admin admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:25.778999 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4b8854-feb3-40b0-8028-0cf253159ac9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:25.791335 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0a42ce-31f8-4637-9f24-5c52f2add462 None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:25.791335 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0a42ce-31f8-4637-9f24-5c52f2add462 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:25.791335 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0a42ce-31f8-4637-9f24-5c52f2add462 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:25.795564 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4b8854-feb3-40b0-8028-0cf253159ac9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:25.796585 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4b8854-feb3-40b0-8028-0cf253159ac9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:41:25.797426 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4b8854-feb3-40b0-8028-0cf253159ac9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:25.799857 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4b8854-feb3-40b0-8028-0cf253159ac9 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:25.851249 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1006/2008] 10.222.0.45 () {58 vars in 1153 bytes} [Wed Aug 24 17:41:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:41:25.852174 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4b8854-feb3-40b0-8028-0cf253159ac9 admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:25.871883 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-5a0a42ce-31f8-4637-9f24-5c52f2add462 None None] MFA Rules not processed for user `3c60d052608a4dce99944f07912a36f2`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:41:25.873267 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-5a0a42ce-31f8-4637-9f24-5c52f2add462 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:25.874293 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4b8854-feb3-40b0-8028-0cf253159ac9 admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:25.895040 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc07c7a-5a9b-4492-96cb-f5adb1950abc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:25.895761 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc07c7a-5a9b-4492-96cb-f5adb1950abc admin admin] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:25.896042 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc07c7a-5a9b-4492-96cb-f5adb1950abc admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:25.896313 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc07c7a-5a9b-4492-96cb-f5adb1950abc admin admin] PATH_INFO: `/v3/projects` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:25.896976 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc07c7a-5a9b-4492-96cb-f5adb1950abc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:25.898904 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc07c7a-5a9b-4492-96cb-f5adb1950abc admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:25.963614 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1007/2009] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 17:41:25 2022] POST /identity/v3/projects => generated 387 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:41:25.964672 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc07c7a-5a9b-4492-96cb-f5adb1950abc admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:26.022893 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1003/2010] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:41:25 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:41:26.040468 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0a42ce-31f8-4637-9f24-5c52f2add462 None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:26.043530 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-5a0a42ce-31f8-4637-9f24-5c52f2add462 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:26.054314 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc07c7a-5a9b-4492-96cb-f5adb1950abc admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:26.062357 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecc9409-831d-4c68-b47c-239d3001a948 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:26.063055 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecc9409-831d-4c68-b47c-239d3001a948 admin admin] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:26.063387 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecc9409-831d-4c68-b47c-239d3001a948 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:26.063725 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecc9409-831d-4c68-b47c-239d3001a948 admin admin] PATH_INFO: `/v3/users` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:26.064479 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecc9409-831d-4c68-b47c-239d3001a948 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:26.066483 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecc9409-831d-4c68-b47c-239d3001a948 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:26.098702 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:41:26.101114 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1008/2011] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 17:41:25 2022] POST /identity/v3/users => generated 294 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:41:26.108889 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecc9409-831d-4c68-b47c-239d3001a948 admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:26.126532 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecc9409-831d-4c68-b47c-239d3001a948 admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:26.132878 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29f277b-3e69-4e51-bee5-634cacffce90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:26.133433 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29f277b-3e69-4e51-bee5-634cacffce90 admin admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:26.133958 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29f277b-3e69-4e51-bee5-634cacffce90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:26.134459 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29f277b-3e69-4e51-bee5-634cacffce90 admin admin] PATH_INFO: `/v3/roles` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:26.135320 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29f277b-3e69-4e51-bee5-634cacffce90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:26.137206 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29f277b-3e69-4e51-bee5-634cacffce90 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:26.156493 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1009/2012] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:41:26 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:41:26.157532 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29f277b-3e69-4e51-bee5-634cacffce90 admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:26.170209 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29f277b-3e69-4e51-bee5-634cacffce90 admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:26.185660 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05063ebd-5d37-40b7-8733-123d208c141e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:26.186828 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05063ebd-5d37-40b7-8733-123d208c141e admin admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:26.187210 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05063ebd-5d37-40b7-8733-123d208c141e admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:26.187587 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05063ebd-5d37-40b7-8733-123d208c141e admin admin] PATH_INFO: `/v3/roles` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:26.188501 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05063ebd-5d37-40b7-8733-123d208c141e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:26.191330 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05063ebd-5d37-40b7-8733-123d208c141e admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:26.203421 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1010/2013] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:41:26 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:41:26.207657 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05063ebd-5d37-40b7-8733-123d208c141e admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:26.216290 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0a42ce-31f8-4637-9f24-5c52f2add462 None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:26.219677 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05063ebd-5d37-40b7-8733-123d208c141e admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:26.226588 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13374e81-4234-4ea1-b1f7-4675d6620624 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:26.227483 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13374e81-4234-4ea1-b1f7-4675d6620624 admin admin] REQUEST_METHOD: `PUT` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:26.227960 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13374e81-4234-4ea1-b1f7-4675d6620624 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:26.228414 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13374e81-4234-4ea1-b1f7-4675d6620624 admin admin] PATH_INFO: `/v3/projects/557369eea1c84593b8089d199ad93698/users/9007ba212c024438b70dbd1300a945c8/roles/02476a5b82eb4b188c18a5d706a3ef68` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:26.247734 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13374e81-4234-4ea1-b1f7-4675d6620624 admin admin] RBAC: Authorizing `identity:create_grant(project_id=557369eea1c84593b8089d199ad93698, user_id=9007ba212c024438b70dbd1300a945c8, role_id=02476a5b82eb4b188c18a5d706a3ef68)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:26.251809 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13374e81-4234-4ea1-b1f7-4675d6620624 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:26.276607 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1011/2014] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 17:41:26 2022] PUT /identity/v3/projects/557369eea1c84593b8089d199ad93698/users/9007ba212c024438b70dbd1300a945c8/roles/02476a5b82eb4b188c18a5d706a3ef68 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:41:26.287210 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33926d7-6de1-42a0-9dcd-66842cdf0fe1 None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:26.287900 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33926d7-6de1-42a0-9dcd-66842cdf0fe1 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:26.288629 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33926d7-6de1-42a0-9dcd-66842cdf0fe1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:26.301218 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15baecb0-9451-4e87-926c-d941e07dde43 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '053b7ce6d6f649ed91a26e434dab805e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:26.302829 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15baecb0-9451-4e87-926c-d941e07dde43 service nova] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:26.303297 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15baecb0-9451-4e87-926c-d941e07dde43 service nova] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:26.303718 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15baecb0-9451-4e87-926c-d941e07dde43 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:26.305964 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-15baecb0-9451-4e87-926c-d941e07dde43 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:26.315678 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:41:26.330010 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-b33926d7-6de1-42a0-9dcd-66842cdf0fe1 None None] MFA Rules not processed for user `9007ba212c024438b70dbd1300a945c8`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:41:26.331235 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-b33926d7-6de1-42a0-9dcd-66842cdf0fe1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:26.408742 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15baecb0-9451-4e87-926c-d941e07dde43 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:26.410818 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15baecb0-9451-4e87-926c-d941e07dde43 service nova] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:26.419567 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1012/2015] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:41:26 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:41:26.420457 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33926d7-6de1-42a0-9dcd-66842cdf0fe1 None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:26.440763 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1004/2016] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:41:26 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:41:26.496160 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33926d7-6de1-42a0-9dcd-66842cdf0fe1 None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:26.501165 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9588db-9cbc-4028-9a39-603ce513d2b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:26.502400 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9588db-9cbc-4028-9a39-603ce513d2b8 admin admin] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:26.502684 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9588db-9cbc-4028-9a39-603ce513d2b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:26.502958 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9588db-9cbc-4028-9a39-603ce513d2b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:26.503583 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9588db-9cbc-4028-9a39-603ce513d2b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:26.505603 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9588db-9cbc-4028-9a39-603ce513d2b8 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:26.538801 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1013/2017] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 17:41:26 2022] POST /identity/v3/projects => generated 387 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:41:26.540352 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15baecb0-9451-4e87-926c-d941e07dde43 service nova] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:26.551387 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15baecb0-9451-4e87-926c-d941e07dde43 service nova] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:26.615724 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c583585b-be1c-4748-a94e-d1a49b82ccec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:26.616580 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c583585b-be1c-4748-a94e-d1a49b82ccec admin admin] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:26.616963 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c583585b-be1c-4748-a94e-d1a49b82ccec admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:26.617255 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c583585b-be1c-4748-a94e-d1a49b82ccec admin admin] PATH_INFO: `/v3/users` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:26.617873 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c583585b-be1c-4748-a94e-d1a49b82ccec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:26.620730 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c583585b-be1c-4748-a94e-d1a49b82ccec admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:26.630921 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:41:26.646973 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9588db-9cbc-4028-9a39-603ce513d2b8 admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:26.652136 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1005/2018] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 17:41:26 2022] POST /identity/v3/users => generated 300 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:41:26.662385 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9588db-9cbc-4028-9a39-603ce513d2b8 admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:26.667738 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e154016-9523-47ad-80a4-0b0763640b21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:26.668530 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e154016-9523-47ad-80a4-0b0763640b21 admin admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:26.668944 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e154016-9523-47ad-80a4-0b0763640b21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:26.669308 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e154016-9523-47ad-80a4-0b0763640b21 admin admin] PATH_INFO: `/v3/roles` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:26.670088 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e154016-9523-47ad-80a4-0b0763640b21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:26.672314 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e154016-9523-47ad-80a4-0b0763640b21 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:26.687585 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1014/2019] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:41:26 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 17:41:26.690292 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c583585b-be1c-4748-a94e-d1a49b82ccec admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:26.700952 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c583585b-be1c-4748-a94e-d1a49b82ccec admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:26.706100 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4405dd-ee77-491b-9016-a36d744de031 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:26.706750 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4405dd-ee77-491b-9016-a36d744de031 admin admin] REQUEST_METHOD: `PUT` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:26.707025 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4405dd-ee77-491b-9016-a36d744de031 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:26.707294 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4405dd-ee77-491b-9016-a36d744de031 admin admin] PATH_INFO: `/v3/projects/36d0f3e8c4df493fb55b7065dcaf0f69/users/5b2b83e498c7432f9d36e4881c4f76c1/roles/e411ad9eb96e4e8d85b59b5b700ada7b` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:26.722539 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4405dd-ee77-491b-9016-a36d744de031 admin admin] RBAC: Authorizing `identity:create_grant(project_id=36d0f3e8c4df493fb55b7065dcaf0f69, user_id=5b2b83e498c7432f9d36e4881c4f76c1, role_id=e411ad9eb96e4e8d85b59b5b700ada7b)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:26.725251 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4405dd-ee77-491b-9016-a36d744de031 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:26.746484 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1006/2020] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 17:41:26 2022] PUT /identity/v3/projects/36d0f3e8c4df493fb55b7065dcaf0f69/users/5b2b83e498c7432f9d36e4881c4f76c1/roles/e411ad9eb96e4e8d85b59b5b700ada7b => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:41:26.751340 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e154016-9523-47ad-80a4-0b0763640b21 admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:26.765370 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e154016-9523-47ad-80a4-0b0763640b21 admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:26.931324 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161df071-288a-47c0-a5c6-10cb2714bfa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:26.932039 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161df071-288a-47c0-a5c6-10cb2714bfa0 admin admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:26.932320 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161df071-288a-47c0-a5c6-10cb2714bfa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:26.932787 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161df071-288a-47c0-a5c6-10cb2714bfa0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:26.933316 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161df071-288a-47c0-a5c6-10cb2714bfa0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:26.934942 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161df071-288a-47c0-a5c6-10cb2714bfa0 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:26.949416 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1015/2021] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:41:26 2022] GET /identity/v3/roles => generated 1420 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:41:26.953566 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4405dd-ee77-491b-9016-a36d744de031 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:26.966491 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4405dd-ee77-491b-9016-a36d744de031 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:26.975040 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ecf3f4-9853-4e43-bac0-e2ff145fe4be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:26.975952 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ecf3f4-9853-4e43-bac0-e2ff145fe4be admin admin] REQUEST_METHOD: `PUT` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:26.976418 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ecf3f4-9853-4e43-bac0-e2ff145fe4be admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:26.976937 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ecf3f4-9853-4e43-bac0-e2ff145fe4be admin admin] PATH_INFO: `/v3/projects/36d0f3e8c4df493fb55b7065dcaf0f69/users/5b2b83e498c7432f9d36e4881c4f76c1/roles/e411ad9eb96e4e8d85b59b5b700ada7b` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:26.979159 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ecf3f4-9853-4e43-bac0-e2ff145fe4be admin admin] RBAC: Authorizing `identity:create_grant(project_id=36d0f3e8c4df493fb55b7065dcaf0f69, user_id=5b2b83e498c7432f9d36e4881c4f76c1, role_id=e411ad9eb96e4e8d85b59b5b700ada7b)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:26.982528 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ecf3f4-9853-4e43-bac0-e2ff145fe4be admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:26.999013 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1007/2022] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 17:41:26 2022] PUT /identity/v3/projects/36d0f3e8c4df493fb55b7065dcaf0f69/users/5b2b83e498c7432f9d36e4881c4f76c1/roles/e411ad9eb96e4e8d85b59b5b700ada7b => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:41:27.011581 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af59ad43-2473-4d4b-ba2b-751836fd2a5c None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:27.012045 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af59ad43-2473-4d4b-ba2b-751836fd2a5c None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:27.012358 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af59ad43-2473-4d4b-ba2b-751836fd2a5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:27.053070 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:41:27.087155 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-af59ad43-2473-4d4b-ba2b-751836fd2a5c None None] MFA Rules not processed for user `5b2b83e498c7432f9d36e4881c4f76c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:41:27.089452 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-af59ad43-2473-4d4b-ba2b-751836fd2a5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:27.229764 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1016/2023] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:41:27 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:41:27.246291 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ecf3f4-9853-4e43-bac0-e2ff145fe4be admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:27.248118 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-88ecf3f4-9853-4e43-bac0-e2ff145fe4be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:27.399230 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ecf3f4-9853-4e43-bac0-e2ff145fe4be admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:27.407999 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeefa905-cb48-48ef-b0df-a051b67afa9e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59be342541264942ad87f60ff4de6c8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:27.409084 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeefa905-cb48-48ef-b0df-a051b67afa9e service cinder] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:27.409403 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeefa905-cb48-48ef-b0df-a051b67afa9e service cinder] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:27.409791 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeefa905-cb48-48ef-b0df-a051b67afa9e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:27.431580 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeefa905-cb48-48ef-b0df-a051b67afa9e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:27.433235 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeefa905-cb48-48ef-b0df-a051b67afa9e service cinder] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:27.450152 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1008/2024] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:41:27 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:41:28.684657 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af59ad43-2473-4d4b-ba2b-751836fd2a5c None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:28.686067 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-af59ad43-2473-4d4b-ba2b-751836fd2a5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:28.703997 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeefa905-cb48-48ef-b0df-a051b67afa9e service cinder] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:28.780234 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af59ad43-2473-4d4b-ba2b-751836fd2a5c None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:28.786904 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e3d382-b37e-44f0-830d-7746995089d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:28.787697 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e3d382-b37e-44f0-830d-7746995089d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:28.788083 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e3d382-b37e-44f0-830d-7746995089d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:28.788447 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e3d382-b37e-44f0-830d-7746995089d2 admin admin] PATH_INFO: `/v3/users/6873bc3749844100a2ad58c1160204bd` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:28.790441 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e3d382-b37e-44f0-830d-7746995089d2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6873bc3749844100a2ad58c1160204bd)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:28.791755 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e3d382-b37e-44f0-830d-7746995089d2 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:28.817963 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeefa905-cb48-48ef-b0df-a051b67afa9e service cinder] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:28.834433 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696628bc-94d4-45ef-b236-c200f108746d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:28.838859 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696628bc-94d4-45ef-b236-c200f108746d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:28.839327 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696628bc-94d4-45ef-b236-c200f108746d admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:28.839794 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696628bc-94d4-45ef-b236-c200f108746d admin admin] PATH_INFO: `/v3/users/9007ba212c024438b70dbd1300a945c8` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:28.842947 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696628bc-94d4-45ef-b236-c200f108746d admin admin] RBAC: Authorizing `identity:delete_user(user_id=9007ba212c024438b70dbd1300a945c8)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:28.851118 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696628bc-94d4-45ef-b236-c200f108746d admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:28.864884 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-86e3d382-b37e-44f0-830d-7746995089d2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6873bc3749844100a2ad58c1160204bd'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:28.865970 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-86e3d382-b37e-44f0-830d-7746995089d2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6873bc3749844100a2ad58c1160204bd'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:28.881897 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-86e3d382-b37e-44f0-830d-7746995089d2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6873bc3749844100a2ad58c1160204bd'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:28.883687 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-86e3d382-b37e-44f0-830d-7746995089d2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6873bc3749844100a2ad58c1160204bd'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:28.912920 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-86e3d382-b37e-44f0-830d-7746995089d2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6873bc3749844100a2ad58c1160204bd'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:28.942648 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1017/2025] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 17:41:28 2022] DELETE /identity/v3/users/6873bc3749844100a2ad58c1160204bd => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:41:28.949149 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eef46af-1615-4e7e-a629-bef930a250c6 None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:28.949517 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eef46af-1615-4e7e-a629-bef930a250c6 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:28.949710 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eef46af-1615-4e7e-a629-bef930a250c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:28.956355 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-696628bc-94d4-45ef-b236-c200f108746d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9007ba212c024438b70dbd1300a945c8'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:28.957320 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-696628bc-94d4-45ef-b236-c200f108746d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9007ba212c024438b70dbd1300a945c8'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:28.991489 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-696628bc-94d4-45ef-b236-c200f108746d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9007ba212c024438b70dbd1300a945c8'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:28.992202 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-696628bc-94d4-45ef-b236-c200f108746d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9007ba212c024438b70dbd1300a945c8'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:29.013931 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:INSERT=4,keystone:SELECT=88,keystone:DELETE=8 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:41:29.019662 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-696628bc-94d4-45ef-b236-c200f108746d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9007ba212c024438b70dbd1300a945c8'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:29.020351 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-1eef46af-1615-4e7e-a629-bef930a250c6 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:41:29.021783 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-1eef46af-1615-4e7e-a629-bef930a250c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:29.031407 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:41:29.059187 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1009/2026] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 17:41:28 2022] DELETE /identity/v3/users/9007ba212c024438b70dbd1300a945c8 => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:41:29.065673 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43a9500-0592-4c19-b677-0ed73fe8be0d None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:29.066281 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43a9500-0592-4c19-b677-0ed73fe8be0d None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:29.066635 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43a9500-0592-4c19-b677-0ed73fe8be0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:29.101718 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-f43a9500-0592-4c19-b677-0ed73fe8be0d None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:41:29.103062 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-f43a9500-0592-4c19-b677-0ed73fe8be0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:29.125072 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1018/2027] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:41:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:41:29.137701 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eef46af-1615-4e7e-a629-bef930a250c6 None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:29.139927 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-1eef46af-1615-4e7e-a629-bef930a250c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:29.196203 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1010/2028] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:41:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:41:29.206418 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43a9500-0592-4c19-b677-0ed73fe8be0d None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:29.212885 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eef46af-1615-4e7e-a629-bef930a250c6 None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:29.279812 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa115e5-40a3-4f32-9383-89381890110c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43ad98a250a144099ccd18378aedd4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:29.280541 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa115e5-40a3-4f32-9383-89381890110c service neutron] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:29.280947 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa115e5-40a3-4f32-9383-89381890110c service neutron] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:29.281299 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa115e5-40a3-4f32-9383-89381890110c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:29.294225 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa115e5-40a3-4f32-9383-89381890110c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:29.295422 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43a9500-0592-4c19-b677-0ed73fe8be0d None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:29.295860 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa115e5-40a3-4f32-9383-89381890110c service neutron] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:29.299585 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d01fbf2-f7f8-4766-a09e-15ab76386fe9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43ad98a250a144099ccd18378aedd4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:29.300210 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d01fbf2-f7f8-4766-a09e-15ab76386fe9 service neutron] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:29.300495 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d01fbf2-f7f8-4766-a09e-15ab76386fe9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:29.300800 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d01fbf2-f7f8-4766-a09e-15ab76386fe9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:29.307810 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1019/2029] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:41:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:41:29.321215 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d01fbf2-f7f8-4766-a09e-15ab76386fe9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:29.322689 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d01fbf2-f7f8-4766-a09e-15ab76386fe9 service neutron] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:29.335477 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1011/2030] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:41:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 17:41:29.845959 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa115e5-40a3-4f32-9383-89381890110c service neutron] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:29.847571 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-2fa115e5-40a3-4f32-9383-89381890110c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:29.854324 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d01fbf2-f7f8-4766-a09e-15ab76386fe9 service neutron] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:29.856063 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-2d01fbf2-f7f8-4766-a09e-15ab76386fe9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:29.877780 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa115e5-40a3-4f32-9383-89381890110c service neutron] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:29.881062 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d01fbf2-f7f8-4766-a09e-15ab76386fe9 service neutron] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:29.886460 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420ddae3-f7be-4726-af74-53be120d8299 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:29.887398 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420ddae3-f7be-4726-af74-53be120d8299 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:29.887532 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc705869-d02a-49de-9ac8-419a94c6a43b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:29.887690 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420ddae3-f7be-4726-af74-53be120d8299 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:29.888098 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc705869-d02a-49de-9ac8-419a94c6a43b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:29.888603 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420ddae3-f7be-4726-af74-53be120d8299 admin admin] PATH_INFO: `/v3/projects/125bccf685c94f6db6cd666681fa1ff3` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:29.888885 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc705869-d02a-49de-9ac8-419a94c6a43b admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:29.889173 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc705869-d02a-49de-9ac8-419a94c6a43b admin admin] PATH_INFO: `/v3/projects/557369eea1c84593b8089d199ad93698` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:29.890339 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc705869-d02a-49de-9ac8-419a94c6a43b admin admin] RBAC: Authorizing `identity:delete_project(project_id=557369eea1c84593b8089d199ad93698)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:29.892293 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc705869-d02a-49de-9ac8-419a94c6a43b admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:29.894744 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420ddae3-f7be-4726-af74-53be120d8299 admin admin] RBAC: Authorizing `identity:delete_project(project_id=125bccf685c94f6db6cd666681fa1ff3)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:29.897563 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420ddae3-f7be-4726-af74-53be120d8299 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:29.955563 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-cc705869-d02a-49de-9ac8-419a94c6a43b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '557369eea1c84593b8089d199ad93698', 'request_id': 'req-cc705869-d02a-49de-9ac8-419a94c6a43b'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:29.959817 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-420ddae3-f7be-4726-af74-53be120d8299 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '125bccf685c94f6db6cd666681fa1ff3', 'request_id': 'req-420ddae3-f7be-4726-af74-53be120d8299'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:29.961947 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-cc705869-d02a-49de-9ac8-419a94c6a43b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '557369eea1c84593b8089d199ad93698', 'request_id': 'req-cc705869-d02a-49de-9ac8-419a94c6a43b'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:29.969250 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-420ddae3-f7be-4726-af74-53be120d8299 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '125bccf685c94f6db6cd666681fa1ff3', 'request_id': 'req-420ddae3-f7be-4726-af74-53be120d8299'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:29.969637 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-cc705869-d02a-49de-9ac8-419a94c6a43b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '557369eea1c84593b8089d199ad93698', 'request_id': 'req-cc705869-d02a-49de-9ac8-419a94c6a43b'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:29.977241 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-420ddae3-f7be-4726-af74-53be120d8299 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '125bccf685c94f6db6cd666681fa1ff3', 'request_id': 'req-420ddae3-f7be-4726-af74-53be120d8299'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:29.982397 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:41:29.987150 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-cc705869-d02a-49de-9ac8-419a94c6a43b admin admin] The token cache is being invalidate because project 557369eea1c84593b8089d199ad93698 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81759) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:41:29.987445 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-cc705869-d02a-49de-9ac8-419a94c6a43b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:29.988080 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-cc705869-d02a-49de-9ac8-419a94c6a43b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:29.989706 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1012/2031] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 17:41:29 2022] DELETE /identity/v3/projects/557369eea1c84593b8089d199ad93698 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:41:29.994565 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc705869-d02a-49de-9ac8-419a94c6a43b admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:29.995134 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-420ddae3-f7be-4726-af74-53be120d8299 admin admin] The token cache is being invalidate because project 125bccf685c94f6db6cd666681fa1ff3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81758) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:41:29.995441 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-420ddae3-f7be-4726-af74-53be120d8299 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:29.996179 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-420ddae3-f7be-4726-af74-53be120d8299 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:29.996879 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-cc705869-d02a-49de-9ac8-419a94c6a43b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:29.997967 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1020/2032] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 17:41:29 2022] DELETE /identity/v3/projects/125bccf685c94f6db6cd666681fa1ff3 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:41:30.001569 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420ddae3-f7be-4726-af74-53be120d8299 admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:30.002998 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-420ddae3-f7be-4726-af74-53be120d8299 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:30.007676 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:41:30.088844 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc705869-d02a-49de-9ac8-419a94c6a43b admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:30.092998 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bd820d-7905-48bc-ac98-920ae9477938 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:30.093649 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bd820d-7905-48bc-ac98-920ae9477938 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:30.093931 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bd820d-7905-48bc-ac98-920ae9477938 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:30.094207 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bd820d-7905-48bc-ac98-920ae9477938 admin admin] PATH_INFO: `/v3/users/5b2b83e498c7432f9d36e4881c4f76c1` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:30.095896 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bd820d-7905-48bc-ac98-920ae9477938 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5b2b83e498c7432f9d36e4881c4f76c1)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:30.096238 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420ddae3-f7be-4726-af74-53be120d8299 admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:30.098108 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bd820d-7905-48bc-ac98-920ae9477938 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:30.102373 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6978b8a-e2af-4c6a-94b8-46ed47975abc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:30.103030 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6978b8a-e2af-4c6a-94b8-46ed47975abc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:30.103482 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6978b8a-e2af-4c6a-94b8-46ed47975abc admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:30.103865 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6978b8a-e2af-4c6a-94b8-46ed47975abc admin admin] PATH_INFO: `/v3/users/8b30c795a59641218c9d3626edb34b8b` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:30.105946 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6978b8a-e2af-4c6a-94b8-46ed47975abc admin admin] RBAC: Authorizing `identity:delete_user(user_id=8b30c795a59641218c9d3626edb34b8b)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:30.107961 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6978b8a-e2af-4c6a-94b8-46ed47975abc admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:30.153644 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-a5bd820d-7905-48bc-ac98-920ae9477938 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5b2b83e498c7432f9d36e4881c4f76c1'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:30.154324 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-a5bd820d-7905-48bc-ac98-920ae9477938 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5b2b83e498c7432f9d36e4881c4f76c1'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:30.162011 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-e6978b8a-e2af-4c6a-94b8-46ed47975abc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8b30c795a59641218c9d3626edb34b8b'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:30.162671 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-a5bd820d-7905-48bc-ac98-920ae9477938 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5b2b83e498c7432f9d36e4881c4f76c1'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:30.163437 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-a5bd820d-7905-48bc-ac98-920ae9477938 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5b2b83e498c7432f9d36e4881c4f76c1'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:30.164472 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-e6978b8a-e2af-4c6a-94b8-46ed47975abc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8b30c795a59641218c9d3626edb34b8b'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:30.173248 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-e6978b8a-e2af-4c6a-94b8-46ed47975abc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8b30c795a59641218c9d3626edb34b8b'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:30.173474 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-a5bd820d-7905-48bc-ac98-920ae9477938 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5b2b83e498c7432f9d36e4881c4f76c1'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:30.174192 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-e6978b8a-e2af-4c6a-94b8-46ed47975abc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8b30c795a59641218c9d3626edb34b8b'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:30.199646 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-e6978b8a-e2af-4c6a-94b8-46ed47975abc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8b30c795a59641218c9d3626edb34b8b'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:30.214919 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1013/2033] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 17:41:29 2022] DELETE /identity/v3/users/5b2b83e498c7432f9d36e4881c4f76c1 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:41:30.217169 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1021/2034] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 17:41:30 2022] DELETE /identity/v3/users/8b30c795a59641218c9d3626edb34b8b => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:41:30.524247 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bd820d-7905-48bc-ac98-920ae9477938 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:30.526365 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-a5bd820d-7905-48bc-ac98-920ae9477938 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:30.607951 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bd820d-7905-48bc-ac98-920ae9477938 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:30.638986 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:41:30.666741 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6978b8a-e2af-4c6a-94b8-46ed47975abc admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:30.668701 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-e6978b8a-e2af-4c6a-94b8-46ed47975abc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:30.678943 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3736c880-b9f6-4fdb-a1fb-a8ce329e0dbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:30.679596 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3736c880-b9f6-4fdb-a1fb-a8ce329e0dbb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:30.679873 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3736c880-b9f6-4fdb-a1fb-a8ce329e0dbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:30.680152 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3736c880-b9f6-4fdb-a1fb-a8ce329e0dbb admin admin] PATH_INFO: `/v3/projects/6f2bd28d060349188c072d5b1ce847b1` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:30.681317 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3736c880-b9f6-4fdb-a1fb-a8ce329e0dbb admin admin] RBAC: Authorizing `identity:delete_project(project_id=6f2bd28d060349188c072d5b1ce847b1)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:30.684285 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3736c880-b9f6-4fdb-a1fb-a8ce329e0dbb admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:30.741687 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-3736c880-b9f6-4fdb-a1fb-a8ce329e0dbb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6f2bd28d060349188c072d5b1ce847b1', 'request_id': 'req-3736c880-b9f6-4fdb-a1fb-a8ce329e0dbb'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:30.744239 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6978b8a-e2af-4c6a-94b8-46ed47975abc admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:30.747784 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-3736c880-b9f6-4fdb-a1fb-a8ce329e0dbb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6f2bd28d060349188c072d5b1ce847b1', 'request_id': 'req-3736c880-b9f6-4fdb-a1fb-a8ce329e0dbb'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:30.756615 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-3736c880-b9f6-4fdb-a1fb-a8ce329e0dbb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6f2bd28d060349188c072d5b1ce847b1', 'request_id': 'req-3736c880-b9f6-4fdb-a1fb-a8ce329e0dbb'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:30.768454 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:41:30.772100 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-3736c880-b9f6-4fdb-a1fb-a8ce329e0dbb admin admin] The token cache is being invalidate because project 6f2bd28d060349188c072d5b1ce847b1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81759) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:41:30.772587 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-3736c880-b9f6-4fdb-a1fb-a8ce329e0dbb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:30.773231 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-3736c880-b9f6-4fdb-a1fb-a8ce329e0dbb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:30.774977 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1014/2035] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 17:41:30 2022] DELETE /identity/v3/projects/6f2bd28d060349188c072d5b1ce847b1 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:41:30.779396 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3736c880-b9f6-4fdb-a1fb-a8ce329e0dbb admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:30.780895 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-3736c880-b9f6-4fdb-a1fb-a8ce329e0dbb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:30.823309 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361c4861-3df2-4b06-af09-73effa871468 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:30.824097 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361c4861-3df2-4b06-af09-73effa871468 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:30.824381 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361c4861-3df2-4b06-af09-73effa871468 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:30.824689 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361c4861-3df2-4b06-af09-73effa871468 admin admin] PATH_INFO: `/v3/projects/36d0f3e8c4df493fb55b7065dcaf0f69` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:30.826443 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361c4861-3df2-4b06-af09-73effa871468 admin admin] RBAC: Authorizing `identity:delete_project(project_id=36d0f3e8c4df493fb55b7065dcaf0f69)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:30.828805 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361c4861-3df2-4b06-af09-73effa871468 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:30.856221 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3736c880-b9f6-4fdb-a1fb-a8ce329e0dbb admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:30.861456 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f1f5e9-cc8c-4a8a-b789-dc7c14941abf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:30.862114 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f1f5e9-cc8c-4a8a-b789-dc7c14941abf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:30.862396 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f1f5e9-cc8c-4a8a-b789-dc7c14941abf admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:30.863520 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f1f5e9-cc8c-4a8a-b789-dc7c14941abf admin admin] PATH_INFO: `/v3/users/7b322976a1694e2b9bcbd10b0f5b8fe6` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:30.864736 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f1f5e9-cc8c-4a8a-b789-dc7c14941abf admin admin] RBAC: Authorizing `identity:delete_user(user_id=7b322976a1694e2b9bcbd10b0f5b8fe6)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:30.867518 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f1f5e9-cc8c-4a8a-b789-dc7c14941abf admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:30.886818 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-361c4861-3df2-4b06-af09-73effa871468 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '36d0f3e8c4df493fb55b7065dcaf0f69', 'request_id': 'req-361c4861-3df2-4b06-af09-73effa871468'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:30.896386 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-361c4861-3df2-4b06-af09-73effa871468 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '36d0f3e8c4df493fb55b7065dcaf0f69', 'request_id': 'req-361c4861-3df2-4b06-af09-73effa871468'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:30.903237 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-361c4861-3df2-4b06-af09-73effa871468 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '36d0f3e8c4df493fb55b7065dcaf0f69', 'request_id': 'req-361c4861-3df2-4b06-af09-73effa871468'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:30.914232 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-361c4861-3df2-4b06-af09-73effa871468 admin admin] The token cache is being invalidate because project 36d0f3e8c4df493fb55b7065dcaf0f69 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81758) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:41:30.914526 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-361c4861-3df2-4b06-af09-73effa871468 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:30.915226 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-361c4861-3df2-4b06-af09-73effa871468 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:30.916772 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1022/2036] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 17:41:30 2022] DELETE /identity/v3/projects/36d0f3e8c4df493fb55b7065dcaf0f69 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:41:30.920151 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-a6f1f5e9-cc8c-4a8a-b789-dc7c14941abf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b322976a1694e2b9bcbd10b0f5b8fe6'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:30.921639 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dd3ceb-f1d8-4a54-b283-53bef9359cbb None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:30.921853 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-a6f1f5e9-cc8c-4a8a-b789-dc7c14941abf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b322976a1694e2b9bcbd10b0f5b8fe6'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:30.922068 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dd3ceb-f1d8-4a54-b283-53bef9359cbb None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:30.922350 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dd3ceb-f1d8-4a54-b283-53bef9359cbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:30.931030 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-a6f1f5e9-cc8c-4a8a-b789-dc7c14941abf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b322976a1694e2b9bcbd10b0f5b8fe6'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:30.931930 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-a6f1f5e9-cc8c-4a8a-b789-dc7c14941abf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b322976a1694e2b9bcbd10b0f5b8fe6'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:30.941328 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-a6f1f5e9-cc8c-4a8a-b789-dc7c14941abf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b322976a1694e2b9bcbd10b0f5b8fe6'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:30.943210 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:41:30.943806 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-66dd3ceb-f1d8-4a54-b283-53bef9359cbb None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:41:30.945060 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-66dd3ceb-f1d8-4a54-b283-53bef9359cbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:30.968263 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1015/2037] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 17:41:30 2022] DELETE /identity/v3/users/7b322976a1694e2b9bcbd10b0f5b8fe6 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:41:31.008147 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1023/2038] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:41:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:41:31.015949 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe3055c-b37b-4b76-a868-980f00b75b84 None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:31.016237 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe3055c-b37b-4b76-a868-980f00b75b84 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:31.016514 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe3055c-b37b-4b76-a868-980f00b75b84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:31.038731 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-7fe3055c-b37b-4b76-a868-980f00b75b84 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:41:31.040265 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-7fe3055c-b37b-4b76-a868-980f00b75b84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:31.104968 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1016/2039] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:41:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:41:31.109072 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66dd3ceb-f1d8-4a54-b283-53bef9359cbb None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:31.119445 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66dd3ceb-f1d8-4a54-b283-53bef9359cbb None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:31.127438 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81848da8-7d8d-425c-a43d-c4085136f687 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:31.128172 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81848da8-7d8d-425c-a43d-c4085136f687 admin admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:31.128465 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81848da8-7d8d-425c-a43d-c4085136f687 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:31.128773 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81848da8-7d8d-425c-a43d-c4085136f687 admin admin] PATH_INFO: `/v3/domains` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:31.129604 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81848da8-7d8d-425c-a43d-c4085136f687 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:41:31.129831 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81848da8-7d8d-425c-a43d-c4085136f687 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:31.131605 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81848da8-7d8d-425c-a43d-c4085136f687 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:31.157341 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1024/2040] 10.222.0.45 () {58 vars in 1153 bytes} [Wed Aug 24 17:41:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:41:31.161985 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe3055c-b37b-4b76-a868-980f00b75b84 None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:31.171101 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe3055c-b37b-4b76-a868-980f00b75b84 None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:31.176349 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18544c8-cf5c-4438-b057-9ba84222d868 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:31.176980 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18544c8-cf5c-4438-b057-9ba84222d868 admin admin] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:31.177311 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18544c8-cf5c-4438-b057-9ba84222d868 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:31.177522 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18544c8-cf5c-4438-b057-9ba84222d868 admin admin] PATH_INFO: `/v3/projects` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:31.178154 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18544c8-cf5c-4438-b057-9ba84222d868 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:31.180066 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18544c8-cf5c-4438-b057-9ba84222d868 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:31.208529 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1017/2041] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 17:41:31 2022] POST /identity/v3/projects => generated 371 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:41:31.212681 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81848da8-7d8d-425c-a43d-c4085136f687 admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:31.223026 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81848da8-7d8d-425c-a43d-c4085136f687 admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:31.258524 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18544c8-cf5c-4438-b057-9ba84222d868 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:31.260361 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-b18544c8-cf5c-4438-b057-9ba84222d868 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:31.269578 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:41:31.290347 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b52de44-dbd1-4daa-90ab-5bcaef557a9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:31.291060 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b52de44-dbd1-4daa-90ab-5bcaef557a9c admin admin] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:31.291231 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b52de44-dbd1-4daa-90ab-5bcaef557a9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:31.291544 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b52de44-dbd1-4daa-90ab-5bcaef557a9c admin admin] PATH_INFO: `/v3/users` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:31.292235 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b52de44-dbd1-4daa-90ab-5bcaef557a9c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:31.294178 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b52de44-dbd1-4daa-90ab-5bcaef557a9c admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:31.319933 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1025/2042] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 17:41:31 2022] POST /identity/v3/users => generated 286 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:41:31.321210 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b52de44-dbd1-4daa-90ab-5bcaef557a9c admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:31.330535 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b52de44-dbd1-4daa-90ab-5bcaef557a9c admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:31.335165 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b994b21d-ff4b-4bbd-a079-fe331349f5f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:31.335508 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18544c8-cf5c-4438-b057-9ba84222d868 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:31.335856 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b994b21d-ff4b-4bbd-a079-fe331349f5f8 admin admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:31.336018 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b994b21d-ff4b-4bbd-a079-fe331349f5f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:31.336310 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b994b21d-ff4b-4bbd-a079-fe331349f5f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:31.336969 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b994b21d-ff4b-4bbd-a079-fe331349f5f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:31.338679 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b994b21d-ff4b-4bbd-a079-fe331349f5f8 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:31.341033 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16349510-8d46-482c-a976-875d5c0fafca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:31.341664 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16349510-8d46-482c-a976-875d5c0fafca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:31.341946 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16349510-8d46-482c-a976-875d5c0fafca admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:31.342223 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16349510-8d46-482c-a976-875d5c0fafca admin admin] PATH_INFO: `/v3/projects/d65e1cd424f24093976f978e5b137e32` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:31.343353 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16349510-8d46-482c-a976-875d5c0fafca admin admin] RBAC: Authorizing `identity:delete_project(project_id=d65e1cd424f24093976f978e5b137e32)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:31.345293 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16349510-8d46-482c-a976-875d5c0fafca admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:31.349729 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1026/2043] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:41:31 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:41:31.353539 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b994b21d-ff4b-4bbd-a079-fe331349f5f8 admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:31.363928 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b994b21d-ff4b-4bbd-a079-fe331349f5f8 admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:31.368575 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74741837-987b-46af-b488-2ed291bdb859 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:31.369156 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74741837-987b-46af-b488-2ed291bdb859 admin admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:31.369438 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74741837-987b-46af-b488-2ed291bdb859 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:31.369713 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74741837-987b-46af-b488-2ed291bdb859 admin admin] PATH_INFO: `/v3/roles` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:31.370343 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74741837-987b-46af-b488-2ed291bdb859 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:31.371910 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74741837-987b-46af-b488-2ed291bdb859 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:31.385407 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1027/2044] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:41:31 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 17:41:31.386609 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74741837-987b-46af-b488-2ed291bdb859 admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:31.394038 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-16349510-8d46-482c-a976-875d5c0fafca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd65e1cd424f24093976f978e5b137e32', 'request_id': 'req-16349510-8d46-482c-a976-875d5c0fafca'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:31.396272 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74741837-987b-46af-b488-2ed291bdb859 admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:31.399447 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-16349510-8d46-482c-a976-875d5c0fafca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd65e1cd424f24093976f978e5b137e32', 'request_id': 'req-16349510-8d46-482c-a976-875d5c0fafca'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:31.401924 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:41:31.407445 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-16349510-8d46-482c-a976-875d5c0fafca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd65e1cd424f24093976f978e5b137e32', 'request_id': 'req-16349510-8d46-482c-a976-875d5c0fafca'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:31.425188 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-16349510-8d46-482c-a976-875d5c0fafca admin admin] The token cache is being invalidate because project d65e1cd424f24093976f978e5b137e32 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81759) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:41:31.425609 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-16349510-8d46-482c-a976-875d5c0fafca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:31.426571 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-16349510-8d46-482c-a976-875d5c0fafca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:31.428948 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1018/2045] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 17:41:31 2022] DELETE /identity/v3/projects/d65e1cd424f24093976f978e5b137e32 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:41:31.434976 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fbd9c0-41d7-4044-a4a1-9880cb5f804b None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:31.435457 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fbd9c0-41d7-4044-a4a1-9880cb5f804b None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:31.436290 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fbd9c0-41d7-4044-a4a1-9880cb5f804b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:31.460224 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-33fbd9c0-41d7-4044-a4a1-9880cb5f804b None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:41:31.461603 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-33fbd9c0-41d7-4044-a4a1-9880cb5f804b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:31.476735 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead7c6bf-eafa-4bdd-a943-6fa206d96e79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:31.477477 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead7c6bf-eafa-4bdd-a943-6fa206d96e79 admin admin] REQUEST_METHOD: `PUT` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:31.477866 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead7c6bf-eafa-4bdd-a943-6fa206d96e79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:31.478144 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead7c6bf-eafa-4bdd-a943-6fa206d96e79 admin admin] PATH_INFO: `/v3/projects/40524faaadec48918fc959ea1953d447/users/7139528b89254717b2df4ae5a10849d4/roles/02476a5b82eb4b188c18a5d706a3ef68` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:31.496638 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead7c6bf-eafa-4bdd-a943-6fa206d96e79 admin admin] RBAC: Authorizing `identity:create_grant(project_id=40524faaadec48918fc959ea1953d447, user_id=7139528b89254717b2df4ae5a10849d4, role_id=02476a5b82eb4b188c18a5d706a3ef68)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:31.499560 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead7c6bf-eafa-4bdd-a943-6fa206d96e79 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:31.518022 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1028/2046] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 17:41:31 2022] PUT /identity/v3/projects/40524faaadec48918fc959ea1953d447/users/7139528b89254717b2df4ae5a10849d4/roles/02476a5b82eb4b188c18a5d706a3ef68 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:41:31.531556 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631c8c33-8175-43fe-8086-0cbbef92cbb3 None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:31.533686 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631c8c33-8175-43fe-8086-0cbbef92cbb3 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:31.533981 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631c8c33-8175-43fe-8086-0cbbef92cbb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:31.548393 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1019/2047] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:41:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:41:31.559014 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ef72bf-4422-4385-9b32-413be54c88b9 None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:31.559553 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ef72bf-4422-4385-9b32-413be54c88b9 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:31.559896 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ef72bf-4422-4385-9b32-413be54c88b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:31.565900 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-631c8c33-8175-43fe-8086-0cbbef92cbb3 None None] MFA Rules not processed for user `7139528b89254717b2df4ae5a10849d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:41:31.567328 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-631c8c33-8175-43fe-8086-0cbbef92cbb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:31.582289 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-86ef72bf-4422-4385-9b32-413be54c88b9 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:41:31.583532 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-86ef72bf-4422-4385-9b32-413be54c88b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:31.654985 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1020/2048] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:41:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:41:31.657642 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1029/2049] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:41:31 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:41:31.659537 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ef72bf-4422-4385-9b32-413be54c88b9 None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:31.669763 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631c8c33-8175-43fe-8086-0cbbef92cbb3 None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:31.671284 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-631c8c33-8175-43fe-8086-0cbbef92cbb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:31.676652 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ef72bf-4422-4385-9b32-413be54c88b9 None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:31.687035 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9559fa28-a24a-4e0e-9c5e-64502be9bbce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:31.687666 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9559fa28-a24a-4e0e-9c5e-64502be9bbce admin admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:31.687949 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9559fa28-a24a-4e0e-9c5e-64502be9bbce admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:31.688226 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9559fa28-a24a-4e0e-9c5e-64502be9bbce admin admin] PATH_INFO: `/v3/domains` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:31.688965 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9559fa28-a24a-4e0e-9c5e-64502be9bbce admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:41:31.689344 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9559fa28-a24a-4e0e-9c5e-64502be9bbce admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:31.690988 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9559fa28-a24a-4e0e-9c5e-64502be9bbce admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:31.703310 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:41:31.713220 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1021/2050] 10.222.0.45 () {58 vars in 1153 bytes} [Wed Aug 24 17:41:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:41:31.717781 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9559fa28-a24a-4e0e-9c5e-64502be9bbce admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:31.731545 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9559fa28-a24a-4e0e-9c5e-64502be9bbce admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:31.737082 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8beef5-9723-4887-a1f1-85c2c6a517a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:31.737696 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8beef5-9723-4887-a1f1-85c2c6a517a9 admin admin] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:31.737975 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8beef5-9723-4887-a1f1-85c2c6a517a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:31.738246 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8beef5-9723-4887-a1f1-85c2c6a517a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:31.739000 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8beef5-9723-4887-a1f1-85c2c6a517a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:31.741203 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8beef5-9723-4887-a1f1-85c2c6a517a9 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:31.744993 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631c8c33-8175-43fe-8086-0cbbef92cbb3 None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:31.749911 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c968bb-c3aa-43da-bf2e-c6216f24f640 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59be342541264942ad87f60ff4de6c8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:31.750642 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c968bb-c3aa-43da-bf2e-c6216f24f640 service cinder] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:31.751020 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c968bb-c3aa-43da-bf2e-c6216f24f640 service cinder] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:31.751389 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c968bb-c3aa-43da-bf2e-c6216f24f640 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:31.766100 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c968bb-c3aa-43da-bf2e-c6216f24f640 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:31.768883 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c968bb-c3aa-43da-bf2e-c6216f24f640 service cinder] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:31.780582 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1022/2051] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 17:41:31 2022] POST /identity/v3/projects => generated 379 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:41:31.781783 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8beef5-9723-4887-a1f1-85c2c6a517a9 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:31.791401 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1030/2052] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:41:31 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 17:41:31.825809 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c968bb-c3aa-43da-bf2e-c6216f24f640 service cinder] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:31.827360 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-64c968bb-c3aa-43da-bf2e-c6216f24f640 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:31.842503 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c968bb-c3aa-43da-bf2e-c6216f24f640 service cinder] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:31.889978 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:DELETE=1 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:41:31.895864 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8beef5-9723-4887-a1f1-85c2c6a517a9 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:31.899371 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0351ab-3604-455a-8130-85503c73193c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:31.899999 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0351ab-3604-455a-8130-85503c73193c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:31.900323 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0351ab-3604-455a-8130-85503c73193c admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:31.900543 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0351ab-3604-455a-8130-85503c73193c admin admin] PATH_INFO: `/v3/users/7139528b89254717b2df4ae5a10849d4` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:31.901833 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0351ab-3604-455a-8130-85503c73193c admin admin] RBAC: Authorizing `identity:delete_user(user_id=7139528b89254717b2df4ae5a10849d4)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:31.902306 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932be079-435e-4559-8e25-e62226cd7d49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:31.903140 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932be079-435e-4559-8e25-e62226cd7d49 admin admin] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:31.903635 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932be079-435e-4559-8e25-e62226cd7d49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:31.903782 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0351ab-3604-455a-8130-85503c73193c admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:31.903964 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932be079-435e-4559-8e25-e62226cd7d49 admin admin] PATH_INFO: `/v3/users` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:31.904915 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932be079-435e-4559-8e25-e62226cd7d49 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:31.906828 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932be079-435e-4559-8e25-e62226cd7d49 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:31.932143 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1023/2053] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 17:41:31 2022] POST /identity/v3/users => generated 290 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:41:31.933270 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932be079-435e-4559-8e25-e62226cd7d49 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:31.944517 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932be079-435e-4559-8e25-e62226cd7d49 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:31.949739 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cf8c52-a732-4e60-8400-83d0bddd5b9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:31.950382 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cf8c52-a732-4e60-8400-83d0bddd5b9b admin admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:31.950665 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cf8c52-a732-4e60-8400-83d0bddd5b9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:31.950944 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cf8c52-a732-4e60-8400-83d0bddd5b9b admin admin] PATH_INFO: `/v3/roles` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:31.951616 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98cf8c52-a732-4e60-8400-83d0bddd5b9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:31.952698 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-2e0351ab-3604-455a-8130-85503c73193c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7139528b89254717b2df4ae5a10849d4'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:31.953230 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98cf8c52-a732-4e60-8400-83d0bddd5b9b admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:31.953363 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-2e0351ab-3604-455a-8130-85503c73193c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7139528b89254717b2df4ae5a10849d4'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:31.961292 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-2e0351ab-3604-455a-8130-85503c73193c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7139528b89254717b2df4ae5a10849d4'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:31.961965 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-2e0351ab-3604-455a-8130-85503c73193c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7139528b89254717b2df4ae5a10849d4'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:31.964182 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1024/2054] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:41:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:41:31.968164 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cf8c52-a732-4e60-8400-83d0bddd5b9b admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:31.969813 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-98cf8c52-a732-4e60-8400-83d0bddd5b9b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:31.974998 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-2e0351ab-3604-455a-8130-85503c73193c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7139528b89254717b2df4ae5a10849d4'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:31.983343 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cf8c52-a732-4e60-8400-83d0bddd5b9b admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:31.991359 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fc9608-24f9-445d-b9e5-ae622f5e6e2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:31.992111 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fc9608-24f9-445d-b9e5-ae622f5e6e2c admin admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:31.992410 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fc9608-24f9-445d-b9e5-ae622f5e6e2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:31.992718 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fc9608-24f9-445d-b9e5-ae622f5e6e2c admin admin] PATH_INFO: `/v3/roles` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:31.993338 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fc9608-24f9-445d-b9e5-ae622f5e6e2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:31.995231 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fc9608-24f9-445d-b9e5-ae622f5e6e2c admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:31.999256 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1031/2055] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 17:41:31 2022] DELETE /identity/v3/users/7139528b89254717b2df4ae5a10849d4 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:41:32.003908 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d04f21f-eb8a-4ae8-99e7-5cfb641cce3d None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:32.004216 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d04f21f-eb8a-4ae8-99e7-5cfb641cce3d None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:32.004516 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d04f21f-eb8a-4ae8-99e7-5cfb641cce3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:32.009627 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1025/2056] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:41:31 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 17:41:32.011252 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fc9608-24f9-445d-b9e5-ae622f5e6e2c admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:32.021645 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fc9608-24f9-445d-b9e5-ae622f5e6e2c admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:32.028586 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-9d04f21f-eb8a-4ae8-99e7-5cfb641cce3d None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:41:32.029797 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-9d04f21f-eb8a-4ae8-99e7-5cfb641cce3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:32.073776 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:41:32.101875 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c0c386-0fff-44d0-9057-7832570d8872 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:32.105073 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c0c386-0fff-44d0-9057-7832570d8872 admin admin] REQUEST_METHOD: `PUT` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:32.105364 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c0c386-0fff-44d0-9057-7832570d8872 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:32.105645 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c0c386-0fff-44d0-9057-7832570d8872 admin admin] PATH_INFO: `/v3/projects/f401438159204379b3037e19bc21622d/users/755a7f7495db4bcdacaa7646216a4ebe/roles/02476a5b82eb4b188c18a5d706a3ef68` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:32.112729 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1032/2057] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:41:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:41:32.123607 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d04f21f-eb8a-4ae8-99e7-5cfb641cce3d None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:32.125165 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-9d04f21f-eb8a-4ae8-99e7-5cfb641cce3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:32.128347 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c0c386-0fff-44d0-9057-7832570d8872 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f401438159204379b3037e19bc21622d, user_id=755a7f7495db4bcdacaa7646216a4ebe, role_id=02476a5b82eb4b188c18a5d706a3ef68)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:32.132116 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c0c386-0fff-44d0-9057-7832570d8872 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:32.162749 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1026/2058] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 17:41:32 2022] PUT /identity/v3/projects/f401438159204379b3037e19bc21622d/users/755a7f7495db4bcdacaa7646216a4ebe/roles/02476a5b82eb4b188c18a5d706a3ef68 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:41:32.174916 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07317efa-9faf-4ff6-a1d7-901a366c4683 None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:32.175123 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07317efa-9faf-4ff6-a1d7-901a366c4683 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:32.175393 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07317efa-9faf-4ff6-a1d7-901a366c4683 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:32.241762 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d04f21f-eb8a-4ae8-99e7-5cfb641cce3d None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:32.242123 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-07317efa-9faf-4ff6-a1d7-901a366c4683 None None] MFA Rules not processed for user `755a7f7495db4bcdacaa7646216a4ebe`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:41:32.242123 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-07317efa-9faf-4ff6-a1d7-901a366c4683 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:32.299149 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2660a26c-8bd6-45d5-9d9b-279eea3d81d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43ad98a250a144099ccd18378aedd4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:32.300134 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2660a26c-8bd6-45d5-9d9b-279eea3d81d6 service neutron] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:32.300418 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2660a26c-8bd6-45d5-9d9b-279eea3d81d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:32.300810 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2660a26c-8bd6-45d5-9d9b-279eea3d81d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:32.331628 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1027/2059] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:41:32 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:41:32.334544 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07317efa-9faf-4ff6-a1d7-901a366c4683 None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:32.390472 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:41:32.427323 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2660a26c-8bd6-45d5-9d9b-279eea3d81d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:32.429188 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2660a26c-8bd6-45d5-9d9b-279eea3d81d6 service neutron] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:32.432801 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07317efa-9faf-4ff6-a1d7-901a366c4683 None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:32.437962 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d994ecce-004d-479a-8ecb-2cd375c0cb42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:32.438665 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d994ecce-004d-479a-8ecb-2cd375c0cb42 admin admin] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:32.440007 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d994ecce-004d-479a-8ecb-2cd375c0cb42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:32.440007 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d994ecce-004d-479a-8ecb-2cd375c0cb42 admin admin] PATH_INFO: `/v3/projects` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:32.440525 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d994ecce-004d-479a-8ecb-2cd375c0cb42 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:32.441456 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1033/2060] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:41:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 17:41:32.442539 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d994ecce-004d-479a-8ecb-2cd375c0cb42 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:32.471013 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1028/2061] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 17:41:32 2022] POST /identity/v3/projects => generated 379 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:41:32.474736 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2660a26c-8bd6-45d5-9d9b-279eea3d81d6 service neutron] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:32.485306 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2660a26c-8bd6-45d5-9d9b-279eea3d81d6 service neutron] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:32.535681 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:41:32.550190 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc4a6b8-2f94-4c02-88cf-3b7bb03d8f8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:32.550810 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc4a6b8-2f94-4c02-88cf-3b7bb03d8f8e admin admin] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:32.551107 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc4a6b8-2f94-4c02-88cf-3b7bb03d8f8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:32.551370 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc4a6b8-2f94-4c02-88cf-3b7bb03d8f8e admin admin] PATH_INFO: `/v3/users` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:32.552043 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc4a6b8-2f94-4c02-88cf-3b7bb03d8f8e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:32.554009 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc4a6b8-2f94-4c02-88cf-3b7bb03d8f8e admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:32.576092 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1034/2062] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 17:41:32 2022] POST /identity/v3/users => generated 290 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:41:32.580221 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d994ecce-004d-479a-8ecb-2cd375c0cb42 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:32.592098 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d994ecce-004d-479a-8ecb-2cd375c0cb42 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:32.597999 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a723b5-a851-4acd-b2e6-63b8559ef549 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:32.598675 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a723b5-a851-4acd-b2e6-63b8559ef549 admin admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:32.598964 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a723b5-a851-4acd-b2e6-63b8559ef549 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:32.599235 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a723b5-a851-4acd-b2e6-63b8559ef549 admin admin] PATH_INFO: `/v3/roles` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:32.599958 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a723b5-a851-4acd-b2e6-63b8559ef549 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:32.601654 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a723b5-a851-4acd-b2e6-63b8559ef549 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:32.615360 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1029/2063] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:41:32 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:41:32.616790 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc4a6b8-2f94-4c02-88cf-3b7bb03d8f8e admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:32.628153 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc4a6b8-2f94-4c02-88cf-3b7bb03d8f8e admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:32.633112 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd25ed3-15eb-42e3-bd03-fdd8d6ecb559 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:32.633719 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd25ed3-15eb-42e3-bd03-fdd8d6ecb559 admin admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:32.634029 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd25ed3-15eb-42e3-bd03-fdd8d6ecb559 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:32.634306 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd25ed3-15eb-42e3-bd03-fdd8d6ecb559 admin admin] PATH_INFO: `/v3/roles` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:32.634956 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd25ed3-15eb-42e3-bd03-fdd8d6ecb559 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:32.637924 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd25ed3-15eb-42e3-bd03-fdd8d6ecb559 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:32.648589 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1035/2064] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:41:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:41:32.652399 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a723b5-a851-4acd-b2e6-63b8559ef549 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:32.662878 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a723b5-a851-4acd-b2e6-63b8559ef549 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:32.667746 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f6482d-24de-4fde-99d8-34e3d9a271dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:32.668404 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f6482d-24de-4fde-99d8-34e3d9a271dc admin admin] REQUEST_METHOD: `PUT` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:32.668725 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f6482d-24de-4fde-99d8-34e3d9a271dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:32.669001 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f6482d-24de-4fde-99d8-34e3d9a271dc admin admin] PATH_INFO: `/v3/projects/27d9bd8c3a65475cac8e3f1dfbb5bc3c/users/b7cc4d1668b043df9512014126d3f993/roles/02476a5b82eb4b188c18a5d706a3ef68` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:32.685473 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f6482d-24de-4fde-99d8-34e3d9a271dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=27d9bd8c3a65475cac8e3f1dfbb5bc3c, user_id=b7cc4d1668b043df9512014126d3f993, role_id=02476a5b82eb4b188c18a5d706a3ef68)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:32.688148 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f6482d-24de-4fde-99d8-34e3d9a271dc admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:32.703608 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1030/2065] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 17:41:32 2022] PUT /identity/v3/projects/27d9bd8c3a65475cac8e3f1dfbb5bc3c/users/b7cc4d1668b043df9512014126d3f993/roles/02476a5b82eb4b188c18a5d706a3ef68 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:41:32.714586 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a32de8-71eb-4b46-b804-97c548cc09bc None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:32.714883 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a32de8-71eb-4b46-b804-97c548cc09bc None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:32.715160 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a32de8-71eb-4b46-b804-97c548cc09bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:32.745705 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-b1a32de8-71eb-4b46-b804-97c548cc09bc None None] MFA Rules not processed for user `b7cc4d1668b043df9512014126d3f993`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:41:32.746878 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-b1a32de8-71eb-4b46-b804-97c548cc09bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:32.791271 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f6482d-24de-4fde-99d8-34e3d9a271dc admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:32.794106 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-53f6482d-24de-4fde-99d8-34e3d9a271dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:32.906262 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1036/2066] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:41:32 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:41:32.910626 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a32de8-71eb-4b46-b804-97c548cc09bc None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:32.979160 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f6482d-24de-4fde-99d8-34e3d9a271dc admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:32.984770 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08e0b03-05de-4b35-a7a5-77c74e87b3b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:32.985414 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08e0b03-05de-4b35-a7a5-77c74e87b3b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:32.985905 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08e0b03-05de-4b35-a7a5-77c74e87b3b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:32.986744 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08e0b03-05de-4b35-a7a5-77c74e87b3b9 admin admin] PATH_INFO: `/v3/projects/40524faaadec48918fc959ea1953d447` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:32.987976 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08e0b03-05de-4b35-a7a5-77c74e87b3b9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=40524faaadec48918fc959ea1953d447)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:32.990577 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08e0b03-05de-4b35-a7a5-77c74e87b3b9 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:33.019046 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a32de8-71eb-4b46-b804-97c548cc09bc None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:33.025794 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8911d9e-0494-4647-b56a-f98779f0b7d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:33.026408 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8911d9e-0494-4647-b56a-f98779f0b7d7 admin admin] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:33.026677 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8911d9e-0494-4647-b56a-f98779f0b7d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:33.028711 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8911d9e-0494-4647-b56a-f98779f0b7d7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:33.028711 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8911d9e-0494-4647-b56a-f98779f0b7d7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:33.030957 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8911d9e-0494-4647-b56a-f98779f0b7d7 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:33.076991 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-b08e0b03-05de-4b35-a7a5-77c74e87b3b9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '40524faaadec48918fc959ea1953d447', 'request_id': 'req-b08e0b03-05de-4b35-a7a5-77c74e87b3b9'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:33.087544 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-b08e0b03-05de-4b35-a7a5-77c74e87b3b9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '40524faaadec48918fc959ea1953d447', 'request_id': 'req-b08e0b03-05de-4b35-a7a5-77c74e87b3b9'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:33.093839 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1037/2067] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 17:41:32 2022] POST /identity/v3/projects => generated 377 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:41:33.097043 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-b08e0b03-05de-4b35-a7a5-77c74e87b3b9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '40524faaadec48918fc959ea1953d447', 'request_id': 'req-b08e0b03-05de-4b35-a7a5-77c74e87b3b9'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:33.098308 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8911d9e-0494-4647-b56a-f98779f0b7d7 admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:33.116004 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-b08e0b03-05de-4b35-a7a5-77c74e87b3b9 admin admin] The token cache is being invalidate because project 40524faaadec48918fc959ea1953d447 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81759) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:41:33.116449 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-b08e0b03-05de-4b35-a7a5-77c74e87b3b9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:33.117280 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-b08e0b03-05de-4b35-a7a5-77c74e87b3b9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:33.119627 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1031/2068] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 17:41:32 2022] DELETE /identity/v3/projects/40524faaadec48918fc959ea1953d447 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:41:33.124300 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1382bc-98f6-4fc7-a7d1-9194df4fc75f None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:33.124667 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1382bc-98f6-4fc7-a7d1-9194df4fc75f None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:33.125529 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1382bc-98f6-4fc7-a7d1-9194df4fc75f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:33.161476 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-9a1382bc-98f6-4fc7-a7d1-9194df4fc75f None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:41:33.169484 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-9a1382bc-98f6-4fc7-a7d1-9194df4fc75f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:33.171132 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:41:33.182214 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:41:33.210238 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8911d9e-0494-4647-b56a-f98779f0b7d7 admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:33.216359 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d30166-b6b4-45fb-996a-2c117496b50f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:33.216560 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d30166-b6b4-45fb-996a-2c117496b50f admin admin] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:33.216879 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d30166-b6b4-45fb-996a-2c117496b50f admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:33.217298 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d30166-b6b4-45fb-996a-2c117496b50f admin admin] PATH_INFO: `/v3/users` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:33.217941 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d30166-b6b4-45fb-996a-2c117496b50f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:33.219964 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d30166-b6b4-45fb-996a-2c117496b50f admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:33.250662 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1038/2069] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 17:41:33 2022] POST /identity/v3/users => generated 295 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:41:33.254476 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d30166-b6b4-45fb-996a-2c117496b50f admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:33.256901 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-01d30166-b6b4-45fb-996a-2c117496b50f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:33.271158 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d30166-b6b4-45fb-996a-2c117496b50f admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:33.275587 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1032/2070] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:41:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:41:33.277322 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aaa74d9-9c28-4f4c-98e5-1e299be5bc29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:33.278037 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaa74d9-9c28-4f4c-98e5-1e299be5bc29 admin admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:33.278387 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaa74d9-9c28-4f4c-98e5-1e299be5bc29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:33.278729 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaa74d9-9c28-4f4c-98e5-1e299be5bc29 admin admin] PATH_INFO: `/v3/roles` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:33.279467 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aaa74d9-9c28-4f4c-98e5-1e299be5bc29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:33.282529 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aaa74d9-9c28-4f4c-98e5-1e299be5bc29 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:33.285407 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f10a71-f77b-4fd8-b2f2-ac881bc3db98 None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:33.285696 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f10a71-f77b-4fd8-b2f2-ac881bc3db98 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:33.285973 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f10a71-f77b-4fd8-b2f2-ac881bc3db98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:33.295394 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1039/2071] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:41:33 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:41:33.299809 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aaa74d9-9c28-4f4c-98e5-1e299be5bc29 admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:33.313474 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-78f10a71-f77b-4fd8-b2f2-ac881bc3db98 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:41:33.314937 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-78f10a71-f77b-4fd8-b2f2-ac881bc3db98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:33.324799 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aaa74d9-9c28-4f4c-98e5-1e299be5bc29 admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:33.330421 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1033/2072] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:41:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:41:33.331448 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f10a71-f77b-4fd8-b2f2-ac881bc3db98 None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:33.336757 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d613eba7-bcb0-4c93-b2ee-143e1552ce23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:33.337451 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d613eba7-bcb0-4c93-b2ee-143e1552ce23 admin admin] REQUEST_METHOD: `PUT` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:33.337737 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d613eba7-bcb0-4c93-b2ee-143e1552ce23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:33.338013 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d613eba7-bcb0-4c93-b2ee-143e1552ce23 admin admin] PATH_INFO: `/v3/projects/872cc9b717f14d509c5100820662fee2/users/467aa4ccb54a42359a93f5ee6a11a2cc/roles/e411ad9eb96e4e8d85b59b5b700ada7b` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:33.342232 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f10a71-f77b-4fd8-b2f2-ac881bc3db98 None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:33.347702 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89709a0-56e7-46d6-beea-16648baca8d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:33.348314 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89709a0-56e7-46d6-beea-16648baca8d4 admin admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:33.349305 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89709a0-56e7-46d6-beea-16648baca8d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:33.349648 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89709a0-56e7-46d6-beea-16648baca8d4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:33.350309 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89709a0-56e7-46d6-beea-16648baca8d4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:41:33.350616 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89709a0-56e7-46d6-beea-16648baca8d4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:33.352225 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89709a0-56e7-46d6-beea-16648baca8d4 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:33.355812 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d613eba7-bcb0-4c93-b2ee-143e1552ce23 admin admin] RBAC: Authorizing `identity:create_grant(project_id=872cc9b717f14d509c5100820662fee2, user_id=467aa4ccb54a42359a93f5ee6a11a2cc, role_id=e411ad9eb96e4e8d85b59b5b700ada7b)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:33.358279 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d613eba7-bcb0-4c93-b2ee-143e1552ce23 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:33.368535 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1034/2073] 10.222.0.45 () {58 vars in 1153 bytes} [Wed Aug 24 17:41:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:41:33.373039 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89709a0-56e7-46d6-beea-16648baca8d4 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:33.379602 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1040/2074] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 17:41:33 2022] PUT /identity/v3/projects/872cc9b717f14d509c5100820662fee2/users/467aa4ccb54a42359a93f5ee6a11a2cc/roles/e411ad9eb96e4e8d85b59b5b700ada7b => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:41:33.383472 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89709a0-56e7-46d6-beea-16648baca8d4 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:33.383655 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d613eba7-bcb0-4c93-b2ee-143e1552ce23 admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:33.487170 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755da361-8277-4a12-a47b-67e1d1150680 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:33.488302 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755da361-8277-4a12-a47b-67e1d1150680 admin admin] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:33.488830 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755da361-8277-4a12-a47b-67e1d1150680 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:33.489335 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755da361-8277-4a12-a47b-67e1d1150680 admin admin] PATH_INFO: `/v3/projects` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:33.490268 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755da361-8277-4a12-a47b-67e1d1150680 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:33.492726 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755da361-8277-4a12-a47b-67e1d1150680 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:33.521279 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d613eba7-bcb0-4c93-b2ee-143e1552ce23 admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:33.528848 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc2b1c4-1e7a-4345-96cd-e03ea90e92f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:33.529867 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc2b1c4-1e7a-4345-96cd-e03ea90e92f8 admin admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:33.530425 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc2b1c4-1e7a-4345-96cd-e03ea90e92f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:33.530842 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc2b1c4-1e7a-4345-96cd-e03ea90e92f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:33.531766 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc2b1c4-1e7a-4345-96cd-e03ea90e92f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:33.534328 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc2b1c4-1e7a-4345-96cd-e03ea90e92f8 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:33.535833 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1035/2075] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 17:41:33 2022] POST /identity/v3/projects => generated 373 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:41:33.541026 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755da361-8277-4a12-a47b-67e1d1150680 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:33.549313 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1041/2076] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:41:33 2022] GET /identity/v3/roles => generated 1420 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:41:33.553392 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc2b1c4-1e7a-4345-96cd-e03ea90e92f8 admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:33.563356 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc2b1c4-1e7a-4345-96cd-e03ea90e92f8 admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:33.613062 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755da361-8277-4a12-a47b-67e1d1150680 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:33.617962 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3db42d-308b-4637-a96c-1657a8da063f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:33.618705 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3db42d-308b-4637-a96c-1657a8da063f admin admin] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:33.619006 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3db42d-308b-4637-a96c-1657a8da063f admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:33.619277 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3db42d-308b-4637-a96c-1657a8da063f admin admin] PATH_INFO: `/v3/users` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:33.619907 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3db42d-308b-4637-a96c-1657a8da063f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:33.621849 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3db42d-308b-4637-a96c-1657a8da063f admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:33.625676 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5ec647-537a-47cb-b4dd-4b40c91e69aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:33.626339 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5ec647-537a-47cb-b4dd-4b40c91e69aa admin admin] REQUEST_METHOD: `PUT` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:33.627172 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5ec647-537a-47cb-b4dd-4b40c91e69aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:33.627434 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5ec647-537a-47cb-b4dd-4b40c91e69aa admin admin] PATH_INFO: `/v3/projects/872cc9b717f14d509c5100820662fee2/users/467aa4ccb54a42359a93f5ee6a11a2cc/roles/e411ad9eb96e4e8d85b59b5b700ada7b` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:33.629413 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5ec647-537a-47cb-b4dd-4b40c91e69aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=872cc9b717f14d509c5100820662fee2, user_id=467aa4ccb54a42359a93f5ee6a11a2cc, role_id=e411ad9eb96e4e8d85b59b5b700ada7b)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:33.632094 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5ec647-537a-47cb-b4dd-4b40c91e69aa admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:33.644566 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1042/2077] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 17:41:33 2022] PUT /identity/v3/projects/872cc9b717f14d509c5100820662fee2/users/467aa4ccb54a42359a93f5ee6a11a2cc/roles/e411ad9eb96e4e8d85b59b5b700ada7b => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:41:33.645161 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1036/2078] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 17:41:33 2022] POST /identity/v3/users => generated 287 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:41:33.649174 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5ec647-537a-47cb-b4dd-4b40c91e69aa admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:33.654638 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941ca127-fd88-4450-a31c-d0755447dbdc None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:33.654918 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941ca127-fd88-4450-a31c-d0755447dbdc None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:33.655192 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941ca127-fd88-4450-a31c-d0755447dbdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:33.678401 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:41:33.680612 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:41:33.689883 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-941ca127-fd88-4450-a31c-d0755447dbdc None None] MFA Rules not processed for user `467aa4ccb54a42359a93f5ee6a11a2cc`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:41:33.691104 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-941ca127-fd88-4450-a31c-d0755447dbdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:33.717276 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5ec647-537a-47cb-b4dd-4b40c91e69aa admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:33.721985 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5ac227-b935-468d-9284-7db488260d82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:33.722588 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5ac227-b935-468d-9284-7db488260d82 admin admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:33.722870 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5ac227-b935-468d-9284-7db488260d82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:33.723141 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5ac227-b935-468d-9284-7db488260d82 admin admin] PATH_INFO: `/v3/roles` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:33.723782 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5ac227-b935-468d-9284-7db488260d82 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:33.725405 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5ac227-b935-468d-9284-7db488260d82 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:33.735099 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1043/2079] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:41:33 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 17:41:33.739240 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5ac227-b935-468d-9284-7db488260d82 admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:33.748986 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5ac227-b935-468d-9284-7db488260d82 admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:33.754264 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71336cbf-9ebb-4074-b3b8-de5d7bc37b99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:33.754896 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71336cbf-9ebb-4074-b3b8-de5d7bc37b99 admin admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:33.755180 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71336cbf-9ebb-4074-b3b8-de5d7bc37b99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:33.755455 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71336cbf-9ebb-4074-b3b8-de5d7bc37b99 admin admin] PATH_INFO: `/v3/roles` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:33.756112 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71336cbf-9ebb-4074-b3b8-de5d7bc37b99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:33.757752 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71336cbf-9ebb-4074-b3b8-de5d7bc37b99 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:33.764143 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1037/2080] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:41:33 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:41:33.769180 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1044/2081] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:41:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 17:41:33.772853 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941ca127-fd88-4450-a31c-d0755447dbdc None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:33.776907 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71336cbf-9ebb-4074-b3b8-de5d7bc37b99 admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:33.778629 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-71336cbf-9ebb-4074-b3b8-de5d7bc37b99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:33.785719 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941ca127-fd88-4450-a31c-d0755447dbdc None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:33.790489 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171a73a0-8d0b-41c2-a4e3-371674ceccdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:33.791125 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171a73a0-8d0b-41c2-a4e3-371674ceccdf admin admin] REQUEST_METHOD: `PUT` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:33.791404 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171a73a0-8d0b-41c2-a4e3-371674ceccdf admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:33.791680 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171a73a0-8d0b-41c2-a4e3-371674ceccdf admin admin] PATH_INFO: `/v3/projects/f45259f9717c4d1db98a468ce1b4fb31/users/655809cc4b4a46e19f5b595fd580e720/roles/02476a5b82eb4b188c18a5d706a3ef68` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:33.808666 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171a73a0-8d0b-41c2-a4e3-371674ceccdf admin admin] RBAC: Authorizing `identity:create_grant(project_id=f45259f9717c4d1db98a468ce1b4fb31, user_id=655809cc4b4a46e19f5b595fd580e720, role_id=02476a5b82eb4b188c18a5d706a3ef68)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:33.811307 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171a73a0-8d0b-41c2-a4e3-371674ceccdf admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:33.827131 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1038/2082] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 17:41:33 2022] PUT /identity/v3/projects/f45259f9717c4d1db98a468ce1b4fb31/users/655809cc4b4a46e19f5b595fd580e720/roles/02476a5b82eb4b188c18a5d706a3ef68 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:41:33.841000 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71336cbf-9ebb-4074-b3b8-de5d7bc37b99 admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:33.843180 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd7e05c-2f27-48bd-94f0-0093618ca937 None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:33.843471 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd7e05c-2f27-48bd-94f0-0093618ca937 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:33.843747 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd7e05c-2f27-48bd-94f0-0093618ca937 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:33.880758 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-9cd7e05c-2f27-48bd-94f0-0093618ca937 None None] MFA Rules not processed for user `655809cc4b4a46e19f5b595fd580e720`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:41:33.881978 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-9cd7e05c-2f27-48bd-94f0-0093618ca937 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:33.900269 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ba7463-bfe0-4644-af5d-9876c3af7815 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59be342541264942ad87f60ff4de6c8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:33.902784 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ba7463-bfe0-4644-af5d-9876c3af7815 service cinder] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:33.903180 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ba7463-bfe0-4644-af5d-9876c3af7815 service cinder] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:33.903525 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ba7463-bfe0-4644-af5d-9876c3af7815 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:33.997679 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1039/2083] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:41:33 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:41:34.002694 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd7e05c-2f27-48bd-94f0-0093618ca937 None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:34.040335 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ba7463-bfe0-4644-af5d-9876c3af7815 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:34.041851 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ba7463-bfe0-4644-af5d-9876c3af7815 service cinder] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:34.079952 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd7e05c-2f27-48bd-94f0-0093618ca937 None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:34.081469 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1045/2084] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:41:33 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 17:41:34.085023 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1efa69-160b-456b-92ec-70bbde49f6d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:34.085636 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1efa69-160b-456b-92ec-70bbde49f6d7 admin admin] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:34.085912 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1efa69-160b-456b-92ec-70bbde49f6d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:34.086185 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1efa69-160b-456b-92ec-70bbde49f6d7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:34.086823 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1efa69-160b-456b-92ec-70bbde49f6d7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:34.089913 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1efa69-160b-456b-92ec-70bbde49f6d7 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:34.096256 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:41:34.139306 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1040/2085] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 17:41:34 2022] POST /identity/v3/projects => generated 373 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:41:34.143421 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ba7463-bfe0-4644-af5d-9876c3af7815 service cinder] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:34.149913 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1efa69-160b-456b-92ec-70bbde49f6d7 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:34.155348 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-3d1efa69-160b-456b-92ec-70bbde49f6d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:34.160147 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ba7463-bfe0-4644-af5d-9876c3af7815 service cinder] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:34.173478 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:41:34.221272 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1efa69-160b-456b-92ec-70bbde49f6d7 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:34.226042 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb15264c-1502-429e-80cc-417e5e1febf1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59be342541264942ad87f60ff4de6c8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:34.226650 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb15264c-1502-429e-80cc-417e5e1febf1 service cinder] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:34.226925 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb15264c-1502-429e-80cc-417e5e1febf1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:34.227194 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb15264c-1502-429e-80cc-417e5e1febf1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:34.229004 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-fb15264c-1502-429e-80cc-417e5e1febf1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:34.229544 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e02f87-ac13-4924-90d1-e129b03550d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:34.230156 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e02f87-ac13-4924-90d1-e129b03550d7 admin admin] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:34.230437 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e02f87-ac13-4924-90d1-e129b03550d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:34.230715 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e02f87-ac13-4924-90d1-e129b03550d7 admin admin] PATH_INFO: `/v3/users` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:34.231356 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e02f87-ac13-4924-90d1-e129b03550d7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:34.233323 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e02f87-ac13-4924-90d1-e129b03550d7 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:34.267766 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1046/2086] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 17:41:34 2022] POST /identity/v3/users => generated 293 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:41:34.268790 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e02f87-ac13-4924-90d1-e129b03550d7 admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:34.281853 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e02f87-ac13-4924-90d1-e129b03550d7 admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:34.289179 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021cc390-0bdc-4586-a190-07128c4c9dec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:34.290149 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021cc390-0bdc-4586-a190-07128c4c9dec admin admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:34.290149 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021cc390-0bdc-4586-a190-07128c4c9dec admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:34.290489 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021cc390-0bdc-4586-a190-07128c4c9dec admin admin] PATH_INFO: `/v3/roles` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:34.291203 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021cc390-0bdc-4586-a190-07128c4c9dec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:34.293789 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021cc390-0bdc-4586-a190-07128c4c9dec admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:34.297347 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb15264c-1502-429e-80cc-417e5e1febf1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:34.299326 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb15264c-1502-429e-80cc-417e5e1febf1 service cinder] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:34.306207 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1047/2087] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:41:34 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:41:34.310169 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021cc390-0bdc-4586-a190-07128c4c9dec admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:34.311175 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1041/2088] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:41:34 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:41:34.327741 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021cc390-0bdc-4586-a190-07128c4c9dec admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:34.337917 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21736ffd-2422-439a-b4e7-702e8b092d28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:34.338652 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21736ffd-2422-439a-b4e7-702e8b092d28 admin admin] REQUEST_METHOD: `PUT` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:34.339026 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21736ffd-2422-439a-b4e7-702e8b092d28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:34.339444 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21736ffd-2422-439a-b4e7-702e8b092d28 admin admin] PATH_INFO: `/v3/projects/d2c08d30a5984e629a4738b0630af4b4/users/fcc852216c804360861d1ee037bf6674/roles/e411ad9eb96e4e8d85b59b5b700ada7b` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:34.365114 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21736ffd-2422-439a-b4e7-702e8b092d28 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d2c08d30a5984e629a4738b0630af4b4, user_id=fcc852216c804360861d1ee037bf6674, role_id=e411ad9eb96e4e8d85b59b5b700ada7b)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:34.372172 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21736ffd-2422-439a-b4e7-702e8b092d28 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:34.400329 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1048/2089] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 17:41:34 2022] PUT /identity/v3/projects/d2c08d30a5984e629a4738b0630af4b4/users/fcc852216c804360861d1ee037bf6674/roles/e411ad9eb96e4e8d85b59b5b700ada7b => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:41:34.400713 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb15264c-1502-429e-80cc-417e5e1febf1 service cinder] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:34.417753 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb15264c-1502-429e-80cc-417e5e1febf1 service cinder] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:34.491264 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86237374-63e5-4521-874e-6809a0a3b8b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:34.492105 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86237374-63e5-4521-874e-6809a0a3b8b7 admin admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:34.492394 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86237374-63e5-4521-874e-6809a0a3b8b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:34.492705 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86237374-63e5-4521-874e-6809a0a3b8b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:34.494244 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86237374-63e5-4521-874e-6809a0a3b8b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:34.496061 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86237374-63e5-4521-874e-6809a0a3b8b7 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:34.508029 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1042/2090] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:41:34 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 17:41:34.512161 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21736ffd-2422-439a-b4e7-702e8b092d28 admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:34.523309 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21736ffd-2422-439a-b4e7-702e8b092d28 admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:34.534173 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b4b7eb-738c-412a-af18-c65f3a7134e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:34.534815 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b4b7eb-738c-412a-af18-c65f3a7134e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:34.535070 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b4b7eb-738c-412a-af18-c65f3a7134e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:34.535341 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b4b7eb-738c-412a-af18-c65f3a7134e0 admin admin] PATH_INFO: `/v3/projects/d2c08d30a5984e629a4738b0630af4b4/users/fcc852216c804360861d1ee037bf6674/roles/e411ad9eb96e4e8d85b59b5b700ada7b` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:34.537371 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b4b7eb-738c-412a-af18-c65f3a7134e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d2c08d30a5984e629a4738b0630af4b4, user_id=fcc852216c804360861d1ee037bf6674, role_id=e411ad9eb96e4e8d85b59b5b700ada7b)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:34.540011 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b4b7eb-738c-412a-af18-c65f3a7134e0 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:34.556464 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1049/2091] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 17:41:34 2022] PUT /identity/v3/projects/d2c08d30a5984e629a4738b0630af4b4/users/fcc852216c804360861d1ee037bf6674/roles/e411ad9eb96e4e8d85b59b5b700ada7b => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:41:34.569712 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1177632f-b690-476a-9883-678984b66863 None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:34.570262 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1177632f-b690-476a-9883-678984b66863 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:34.570715 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1177632f-b690-476a-9883-678984b66863 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:34.607156 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-1177632f-b690-476a-9883-678984b66863 None None] MFA Rules not processed for user `fcc852216c804360861d1ee037bf6674`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:41:34.608934 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-1177632f-b690-476a-9883-678984b66863 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:34.651436 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:41:34.750223 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1043/2092] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:41:34 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:41:34.763932 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b4b7eb-738c-412a-af18-c65f3a7134e0 admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:34.845862 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b4b7eb-738c-412a-af18-c65f3a7134e0 admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:34.849768 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7611ab11-d931-469a-872a-a1ff0aece9a9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59be342541264942ad87f60ff4de6c8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:34.850377 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7611ab11-d931-469a-872a-a1ff0aece9a9 service cinder] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:34.850659 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7611ab11-d931-469a-872a-a1ff0aece9a9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:34.850939 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7611ab11-d931-469a-872a-a1ff0aece9a9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:34.906687 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7611ab11-d931-469a-872a-a1ff0aece9a9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:34.908154 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7611ab11-d931-469a-872a-a1ff0aece9a9 service cinder] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:34.916896 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1050/2093] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:41:34 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:41:37.459832 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1177632f-b690-476a-9883-678984b66863 None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:37.537982 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1177632f-b690-476a-9883-678984b66863 None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:37.543595 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4a2ca7-ec09-4492-ad83-7a0087ea4f46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:37.544351 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4a2ca7-ec09-4492-ad83-7a0087ea4f46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:37.544735 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4a2ca7-ec09-4492-ad83-7a0087ea4f46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:37.545083 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4a2ca7-ec09-4492-ad83-7a0087ea4f46 admin admin] PATH_INFO: `/v3/users/755a7f7495db4bcdacaa7646216a4ebe` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:37.546395 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4a2ca7-ec09-4492-ad83-7a0087ea4f46 admin admin] RBAC: Authorizing `identity:delete_user(user_id=755a7f7495db4bcdacaa7646216a4ebe)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:37.548431 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4a2ca7-ec09-4492-ad83-7a0087ea4f46 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:37.602918 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-6a4a2ca7-ec09-4492-ad83-7a0087ea4f46 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '755a7f7495db4bcdacaa7646216a4ebe'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:37.603599 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-6a4a2ca7-ec09-4492-ad83-7a0087ea4f46 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '755a7f7495db4bcdacaa7646216a4ebe'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:37.611220 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-6a4a2ca7-ec09-4492-ad83-7a0087ea4f46 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '755a7f7495db4bcdacaa7646216a4ebe'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:37.611981 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-6a4a2ca7-ec09-4492-ad83-7a0087ea4f46 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '755a7f7495db4bcdacaa7646216a4ebe'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:37.622119 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-6a4a2ca7-ec09-4492-ad83-7a0087ea4f46 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '755a7f7495db4bcdacaa7646216a4ebe'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:37.643715 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1044/2094] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 17:41:37 2022] DELETE /identity/v3/users/755a7f7495db4bcdacaa7646216a4ebe => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:41:37.648078 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3b2fe5-9fa9-4fe9-9ddc-ab5e65fcb551 None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:37.648629 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3b2fe5-9fa9-4fe9-9ddc-ab5e65fcb551 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:37.649047 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3b2fe5-9fa9-4fe9-9ddc-ab5e65fcb551 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:37.678161 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-3a3b2fe5-9fa9-4fe9-9ddc-ab5e65fcb551 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:41:37.679394 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-3a3b2fe5-9fa9-4fe9-9ddc-ab5e65fcb551 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:37.688147 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:41:37.791870 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1051/2095] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:41:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:41:37.824527 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4a2ca7-ec09-4492-ad83-7a0087ea4f46 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:37.826206 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-6a4a2ca7-ec09-4492-ad83-7a0087ea4f46 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:37.895310 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:41:37.934651 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4a2ca7-ec09-4492-ad83-7a0087ea4f46 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:37.939958 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1176da9-8a18-4d87-8d03-9f6c379bfecc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43ad98a250a144099ccd18378aedd4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:37.940898 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1176da9-8a18-4d87-8d03-9f6c379bfecc service neutron] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:37.941379 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1176da9-8a18-4d87-8d03-9f6c379bfecc service neutron] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:37.941879 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1176da9-8a18-4d87-8d03-9f6c379bfecc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:37.954589 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1176da9-8a18-4d87-8d03-9f6c379bfecc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:37.956435 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1176da9-8a18-4d87-8d03-9f6c379bfecc service neutron] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:37.973185 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1045/2096] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:41:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:41:38.124668 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3b2fe5-9fa9-4fe9-9ddc-ab5e65fcb551 None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:38.126126 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-3a3b2fe5-9fa9-4fe9-9ddc-ab5e65fcb551 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:38.142854 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3b2fe5-9fa9-4fe9-9ddc-ab5e65fcb551 None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:38.154188 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661a01b7-afb5-4384-972d-d3ff1c2503eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:38.157488 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661a01b7-afb5-4384-972d-d3ff1c2503eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:38.158562 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661a01b7-afb5-4384-972d-d3ff1c2503eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:38.159102 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661a01b7-afb5-4384-972d-d3ff1c2503eb admin admin] PATH_INFO: `/v3/users/655809cc4b4a46e19f5b595fd580e720` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:38.160822 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661a01b7-afb5-4384-972d-d3ff1c2503eb admin admin] RBAC: Authorizing `identity:delete_user(user_id=655809cc4b4a46e19f5b595fd580e720)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:38.164317 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661a01b7-afb5-4384-972d-d3ff1c2503eb admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:38.271225 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-661a01b7-afb5-4384-972d-d3ff1c2503eb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '655809cc4b4a46e19f5b595fd580e720'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:38.272371 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-661a01b7-afb5-4384-972d-d3ff1c2503eb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '655809cc4b4a46e19f5b595fd580e720'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:38.281775 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-661a01b7-afb5-4384-972d-d3ff1c2503eb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '655809cc4b4a46e19f5b595fd580e720'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:38.283514 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-661a01b7-afb5-4384-972d-d3ff1c2503eb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '655809cc4b4a46e19f5b595fd580e720'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:38.312829 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-661a01b7-afb5-4384-972d-d3ff1c2503eb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '655809cc4b4a46e19f5b595fd580e720'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:38.358312 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1052/2097] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 17:41:38 2022] DELETE /identity/v3/users/655809cc4b4a46e19f5b595fd580e720 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:41:38.363756 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a21e6b-4eb8-4bb5-a2de-fd674ad41699 None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:38.364335 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a21e6b-4eb8-4bb5-a2de-fd674ad41699 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:38.364793 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a21e6b-4eb8-4bb5-a2de-fd674ad41699 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:38.391908 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-68a21e6b-4eb8-4bb5-a2de-fd674ad41699 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:41:38.393865 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-68a21e6b-4eb8-4bb5-a2de-fd674ad41699 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:38.498625 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1046/2098] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:41:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:41:38.515542 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661a01b7-afb5-4384-972d-d3ff1c2503eb admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:38.517107 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-661a01b7-afb5-4384-972d-d3ff1c2503eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:38.598329 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661a01b7-afb5-4384-972d-d3ff1c2503eb admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:38.604190 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55abbdb8-682d-464b-987c-4c104511ffba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43ad98a250a144099ccd18378aedd4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:38.605208 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55abbdb8-682d-464b-987c-4c104511ffba service neutron] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:38.605679 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55abbdb8-682d-464b-987c-4c104511ffba service neutron] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:38.606118 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55abbdb8-682d-464b-987c-4c104511ffba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:38.617841 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55abbdb8-682d-464b-987c-4c104511ffba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:38.619866 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55abbdb8-682d-464b-987c-4c104511ffba service neutron] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:38.623699 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a21e6b-4eb8-4bb5-a2de-fd674ad41699 None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:38.625401 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-68a21e6b-4eb8-4bb5-a2de-fd674ad41699 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:38.633501 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1053/2099] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:41:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:41:38.641455 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a21e6b-4eb8-4bb5-a2de-fd674ad41699 None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:38.647445 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1783e5a-641a-4410-ab99-9fcf6f816219 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:38.648215 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1783e5a-641a-4410-ab99-9fcf6f816219 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:38.648626 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1783e5a-641a-4410-ab99-9fcf6f816219 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:38.649006 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1783e5a-641a-4410-ab99-9fcf6f816219 admin admin] PATH_INFO: `/v3/projects/f401438159204379b3037e19bc21622d` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:38.650259 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1783e5a-641a-4410-ab99-9fcf6f816219 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f401438159204379b3037e19bc21622d)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:38.652287 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1783e5a-641a-4410-ab99-9fcf6f816219 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:38.735068 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-e1783e5a-641a-4410-ab99-9fcf6f816219 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f401438159204379b3037e19bc21622d', 'request_id': 'req-e1783e5a-641a-4410-ab99-9fcf6f816219'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:38.743568 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-e1783e5a-641a-4410-ab99-9fcf6f816219 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f401438159204379b3037e19bc21622d', 'request_id': 'req-e1783e5a-641a-4410-ab99-9fcf6f816219'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:38.752397 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-e1783e5a-641a-4410-ab99-9fcf6f816219 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f401438159204379b3037e19bc21622d', 'request_id': 'req-e1783e5a-641a-4410-ab99-9fcf6f816219'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:38.775967 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-e1783e5a-641a-4410-ab99-9fcf6f816219 admin admin] The token cache is being invalidate because project f401438159204379b3037e19bc21622d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81759) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:41:38.777455 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-e1783e5a-641a-4410-ab99-9fcf6f816219 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:38.780003 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-e1783e5a-641a-4410-ab99-9fcf6f816219 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:38.782320 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1047/2100] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 17:41:38 2022] DELETE /identity/v3/projects/f401438159204379b3037e19bc21622d => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:41:38.787090 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55abbdb8-682d-464b-987c-4c104511ffba service neutron] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:38.798039 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55abbdb8-682d-464b-987c-4c104511ffba service neutron] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:38.830766 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:41:38.882658 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a136f5-4fc5-4f08-98d8-aa9e778a75e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:38.883368 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a136f5-4fc5-4f08-98d8-aa9e778a75e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:38.884660 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a136f5-4fc5-4f08-98d8-aa9e778a75e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:38.884660 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a136f5-4fc5-4f08-98d8-aa9e778a75e7 admin admin] PATH_INFO: `/v3/users/b7cc4d1668b043df9512014126d3f993` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:38.886407 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a136f5-4fc5-4f08-98d8-aa9e778a75e7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b7cc4d1668b043df9512014126d3f993)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:38.888414 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a136f5-4fc5-4f08-98d8-aa9e778a75e7 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:38.961514 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-07a136f5-4fc5-4f08-98d8-aa9e778a75e7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b7cc4d1668b043df9512014126d3f993'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:38.963299 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-07a136f5-4fc5-4f08-98d8-aa9e778a75e7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b7cc4d1668b043df9512014126d3f993'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:38.997231 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-07a136f5-4fc5-4f08-98d8-aa9e778a75e7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b7cc4d1668b043df9512014126d3f993'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:39.000250 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-07a136f5-4fc5-4f08-98d8-aa9e778a75e7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b7cc4d1668b043df9512014126d3f993'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:39.044193 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-07a136f5-4fc5-4f08-98d8-aa9e778a75e7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b7cc4d1668b043df9512014126d3f993'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:39.081402 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1054/2101] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 17:41:38 2022] DELETE /identity/v3/users/b7cc4d1668b043df9512014126d3f993 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:41:39.292470 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1783e5a-641a-4410-ab99-9fcf6f816219 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:39.293076 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-e1783e5a-641a-4410-ab99-9fcf6f816219 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:39.318135 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1783e5a-641a-4410-ab99-9fcf6f816219 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:39.480842 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8754f899-b7cd-4ec2-b43e-d98479ee0a51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:39.481524 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8754f899-b7cd-4ec2-b43e-d98479ee0a51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:39.481804 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8754f899-b7cd-4ec2-b43e-d98479ee0a51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:39.482076 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8754f899-b7cd-4ec2-b43e-d98479ee0a51 admin admin] PATH_INFO: `/v3/projects/f45259f9717c4d1db98a468ce1b4fb31` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:39.485110 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8754f899-b7cd-4ec2-b43e-d98479ee0a51 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f45259f9717c4d1db98a468ce1b4fb31)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:39.485772 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8754f899-b7cd-4ec2-b43e-d98479ee0a51 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:39.510728 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:41:39.517879 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a136f5-4fc5-4f08-98d8-aa9e778a75e7 admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:39.520509 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-07a136f5-4fc5-4f08-98d8-aa9e778a75e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:39.545972 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a136f5-4fc5-4f08-98d8-aa9e778a75e7 admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:39.558169 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b659f1-d409-4445-a104-831f8fc0cedf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:39.558873 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b659f1-d409-4445-a104-831f8fc0cedf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:39.561147 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b659f1-d409-4445-a104-831f8fc0cedf admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:39.561449 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b659f1-d409-4445-a104-831f8fc0cedf admin admin] PATH_INFO: `/v3/projects/27d9bd8c3a65475cac8e3f1dfbb5bc3c` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:39.562965 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b659f1-d409-4445-a104-831f8fc0cedf admin admin] RBAC: Authorizing `identity:delete_project(project_id=27d9bd8c3a65475cac8e3f1dfbb5bc3c)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:39.570793 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b659f1-d409-4445-a104-831f8fc0cedf admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:39.583300 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-8754f899-b7cd-4ec2-b43e-d98479ee0a51 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f45259f9717c4d1db98a468ce1b4fb31', 'request_id': 'req-8754f899-b7cd-4ec2-b43e-d98479ee0a51'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:39.593507 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-8754f899-b7cd-4ec2-b43e-d98479ee0a51 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f45259f9717c4d1db98a468ce1b4fb31', 'request_id': 'req-8754f899-b7cd-4ec2-b43e-d98479ee0a51'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:39.610391 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-8754f899-b7cd-4ec2-b43e-d98479ee0a51 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f45259f9717c4d1db98a468ce1b4fb31', 'request_id': 'req-8754f899-b7cd-4ec2-b43e-d98479ee0a51'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:39.638004 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-8754f899-b7cd-4ec2-b43e-d98479ee0a51 admin admin] The token cache is being invalidate because project f45259f9717c4d1db98a468ce1b4fb31 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81759) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:41:39.639246 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-8754f899-b7cd-4ec2-b43e-d98479ee0a51 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:39.641921 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-8754f899-b7cd-4ec2-b43e-d98479ee0a51 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:39.643985 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1048/2102] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 17:41:39 2022] DELETE /identity/v3/projects/f45259f9717c4d1db98a468ce1b4fb31 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:41:39.651731 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-e6b659f1-d409-4445-a104-831f8fc0cedf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '27d9bd8c3a65475cac8e3f1dfbb5bc3c', 'request_id': 'req-e6b659f1-d409-4445-a104-831f8fc0cedf'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:39.656519 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8754f899-b7cd-4ec2-b43e-d98479ee0a51 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:39.659866 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-8754f899-b7cd-4ec2-b43e-d98479ee0a51 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:39.663013 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-e6b659f1-d409-4445-a104-831f8fc0cedf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '27d9bd8c3a65475cac8e3f1dfbb5bc3c', 'request_id': 'req-e6b659f1-d409-4445-a104-831f8fc0cedf'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:39.678842 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-e6b659f1-d409-4445-a104-831f8fc0cedf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '27d9bd8c3a65475cac8e3f1dfbb5bc3c', 'request_id': 'req-e6b659f1-d409-4445-a104-831f8fc0cedf'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:39.706972 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-e6b659f1-d409-4445-a104-831f8fc0cedf admin admin] The token cache is being invalidate because project 27d9bd8c3a65475cac8e3f1dfbb5bc3c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81758) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:41:39.708891 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-e6b659f1-d409-4445-a104-831f8fc0cedf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:39.709911 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-e6b659f1-d409-4445-a104-831f8fc0cedf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:39.715731 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1055/2103] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 17:41:39 2022] DELETE /identity/v3/projects/27d9bd8c3a65475cac8e3f1dfbb5bc3c => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:41:39.719724 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b659f1-d409-4445-a104-831f8fc0cedf admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:39.725399 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-e6b659f1-d409-4445-a104-831f8fc0cedf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:39.757988 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8754f899-b7cd-4ec2-b43e-d98479ee0a51 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:39.771623 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:41:39.820886 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b659f1-d409-4445-a104-831f8fc0cedf admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:39.824725 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6330cfc8-3fb8-423f-904e-1f025af279c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:39.825740 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6330cfc8-3fb8-423f-904e-1f025af279c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:39.826121 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b1953d-4798-4cac-b0d3-ce5f36e2abcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:39.826260 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6330cfc8-3fb8-423f-904e-1f025af279c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:39.826643 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6330cfc8-3fb8-423f-904e-1f025af279c9 admin admin] PATH_INFO: `/v3/users/fcc852216c804360861d1ee037bf6674` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:39.826769 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b1953d-4798-4cac-b0d3-ce5f36e2abcd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:39.827030 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b1953d-4798-4cac-b0d3-ce5f36e2abcd admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:39.827305 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b1953d-4798-4cac-b0d3-ce5f36e2abcd admin admin] PATH_INFO: `/v3/users/467aa4ccb54a42359a93f5ee6a11a2cc` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:39.828317 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6330cfc8-3fb8-423f-904e-1f025af279c9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fcc852216c804360861d1ee037bf6674)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:39.828459 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b1953d-4798-4cac-b0d3-ce5f36e2abcd admin admin] RBAC: Authorizing `identity:delete_user(user_id=467aa4ccb54a42359a93f5ee6a11a2cc)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:39.830519 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6330cfc8-3fb8-423f-904e-1f025af279c9 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:39.830661 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b1953d-4798-4cac-b0d3-ce5f36e2abcd admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:39.910989 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-75b1953d-4798-4cac-b0d3-ce5f36e2abcd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '467aa4ccb54a42359a93f5ee6a11a2cc'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:39.912044 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-75b1953d-4798-4cac-b0d3-ce5f36e2abcd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '467aa4ccb54a42359a93f5ee6a11a2cc'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:39.913038 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-6330cfc8-3fb8-423f-904e-1f025af279c9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fcc852216c804360861d1ee037bf6674'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:39.913795 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-6330cfc8-3fb8-423f-904e-1f025af279c9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fcc852216c804360861d1ee037bf6674'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:39.915528 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:41:39.921793 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-75b1953d-4798-4cac-b0d3-ce5f36e2abcd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '467aa4ccb54a42359a93f5ee6a11a2cc'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:39.922924 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-75b1953d-4798-4cac-b0d3-ce5f36e2abcd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '467aa4ccb54a42359a93f5ee6a11a2cc'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:39.937001 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-6330cfc8-3fb8-423f-904e-1f025af279c9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fcc852216c804360861d1ee037bf6674'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:39.937001 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-6330cfc8-3fb8-423f-904e-1f025af279c9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fcc852216c804360861d1ee037bf6674'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:39.943995 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-75b1953d-4798-4cac-b0d3-ce5f36e2abcd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '467aa4ccb54a42359a93f5ee6a11a2cc'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:39.952241 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-6330cfc8-3fb8-423f-904e-1f025af279c9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fcc852216c804360861d1ee037bf6674'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:39.968505 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1056/2104] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 17:41:39 2022] DELETE /identity/v3/users/467aa4ccb54a42359a93f5ee6a11a2cc => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:41:40.003434 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1049/2105] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 17:41:39 2022] DELETE /identity/v3/users/fcc852216c804360861d1ee037bf6674 => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:41:40.391401 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b1953d-4798-4cac-b0d3-ce5f36e2abcd admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:40.394191 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-75b1953d-4798-4cac-b0d3-ce5f36e2abcd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:40.431592 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6330cfc8-3fb8-423f-904e-1f025af279c9 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:40.433434 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-6330cfc8-3fb8-423f-904e-1f025af279c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:40.489217 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b1953d-4798-4cac-b0d3-ce5f36e2abcd admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:40.531741 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6330cfc8-3fb8-423f-904e-1f025af279c9 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:40.537751 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cc4fb0-9b23-4e5b-953d-24b644319134 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:40.538394 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cc4fb0-9b23-4e5b-953d-24b644319134 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:40.538676 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cc4fb0-9b23-4e5b-953d-24b644319134 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:40.538950 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cc4fb0-9b23-4e5b-953d-24b644319134 admin admin] PATH_INFO: `/v3/projects/d2c08d30a5984e629a4738b0630af4b4` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:40.540090 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cc4fb0-9b23-4e5b-953d-24b644319134 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d2c08d30a5984e629a4738b0630af4b4)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:40.542524 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cc4fb0-9b23-4e5b-953d-24b644319134 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:40.555181 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:41:40.567228 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a338b74-9f89-4ea8-a786-9aee289ce9d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:40.567896 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a338b74-9f89-4ea8-a786-9aee289ce9d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:40.568306 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a338b74-9f89-4ea8-a786-9aee289ce9d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:40.568613 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a338b74-9f89-4ea8-a786-9aee289ce9d5 admin admin] PATH_INFO: `/v3/projects/872cc9b717f14d509c5100820662fee2` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:40.570997 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a338b74-9f89-4ea8-a786-9aee289ce9d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=872cc9b717f14d509c5100820662fee2)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:40.573517 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a338b74-9f89-4ea8-a786-9aee289ce9d5 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:40.600334 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-33cc4fb0-9b23-4e5b-953d-24b644319134 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd2c08d30a5984e629a4738b0630af4b4', 'request_id': 'req-33cc4fb0-9b23-4e5b-953d-24b644319134'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:40.607203 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-33cc4fb0-9b23-4e5b-953d-24b644319134 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd2c08d30a5984e629a4738b0630af4b4', 'request_id': 'req-33cc4fb0-9b23-4e5b-953d-24b644319134'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:40.616318 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-33cc4fb0-9b23-4e5b-953d-24b644319134 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd2c08d30a5984e629a4738b0630af4b4', 'request_id': 'req-33cc4fb0-9b23-4e5b-953d-24b644319134'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:40.625355 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-9a338b74-9f89-4ea8-a786-9aee289ce9d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '872cc9b717f14d509c5100820662fee2', 'request_id': 'req-9a338b74-9f89-4ea8-a786-9aee289ce9d5'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:40.632914 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-33cc4fb0-9b23-4e5b-953d-24b644319134 admin admin] The token cache is being invalidate because project d2c08d30a5984e629a4738b0630af4b4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81759) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:41:40.633406 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-33cc4fb0-9b23-4e5b-953d-24b644319134 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:40.633636 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-9a338b74-9f89-4ea8-a786-9aee289ce9d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '872cc9b717f14d509c5100820662fee2', 'request_id': 'req-9a338b74-9f89-4ea8-a786-9aee289ce9d5'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:40.634108 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-33cc4fb0-9b23-4e5b-953d-24b644319134 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:40.639217 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1050/2106] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 17:41:40 2022] DELETE /identity/v3/projects/d2c08d30a5984e629a4738b0630af4b4 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:41:40.640803 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bca9845-4239-459a-92f0-d6b0fee1e12c None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:40.640916 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-9a338b74-9f89-4ea8-a786-9aee289ce9d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '872cc9b717f14d509c5100820662fee2', 'request_id': 'req-9a338b74-9f89-4ea8-a786-9aee289ce9d5'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:40.641177 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bca9845-4239-459a-92f0-d6b0fee1e12c None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:40.641459 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bca9845-4239-459a-92f0-d6b0fee1e12c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:40.653244 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-9a338b74-9f89-4ea8-a786-9aee289ce9d5 admin admin] The token cache is being invalidate because project 872cc9b717f14d509c5100820662fee2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81758) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:41:40.653558 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-9a338b74-9f89-4ea8-a786-9aee289ce9d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:40.654375 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-9a338b74-9f89-4ea8-a786-9aee289ce9d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:40.656009 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1057/2107] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 17:41:40 2022] DELETE /identity/v3/projects/872cc9b717f14d509c5100820662fee2 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:41:40.661720 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2da78e6-c987-435e-85ee-f459902bed06 None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:40.662006 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2da78e6-c987-435e-85ee-f459902bed06 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:40.662282 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2da78e6-c987-435e-85ee-f459902bed06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:40.668607 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-0bca9845-4239-459a-92f0-d6b0fee1e12c None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:41:40.669821 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-0bca9845-4239-459a-92f0-d6b0fee1e12c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:40.687400 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-f2da78e6-c987-435e-85ee-f459902bed06 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:41:40.689061 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-f2da78e6-c987-435e-85ee-f459902bed06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:40.726582 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:41:40.743071 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1051/2108] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:41:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:41:40.751598 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a9cc76-ccc1-4b39-a26d-d660e2421f20 None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:40.752621 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a9cc76-ccc1-4b39-a26d-d660e2421f20 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:40.756897 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a9cc76-ccc1-4b39-a26d-d660e2421f20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:40.768596 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1058/2109] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:41:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:41:40.778976 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baabb25-96c4-4273-99b3-cd7582ef33a6 None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:40.779333 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baabb25-96c4-4273-99b3-cd7582ef33a6 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:40.779684 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baabb25-96c4-4273-99b3-cd7582ef33a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:40.791819 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-a0a9cc76-ccc1-4b39-a26d-d660e2421f20 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:41:40.793102 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-a0a9cc76-ccc1-4b39-a26d-d660e2421f20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:40.805177 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1052/2110] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:41:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:41:40.805916 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-6baabb25-96c4-4273-99b3-cd7582ef33a6 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:41:40.805916 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-6baabb25-96c4-4273-99b3-cd7582ef33a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:40.812846 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1059/2111] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:41:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:41:40.816960 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a9cc76-ccc1-4b39-a26d-d660e2421f20 None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:40.817684 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6baabb25-96c4-4273-99b3-cd7582ef33a6 None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:40.826494 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a9cc76-ccc1-4b39-a26d-d660e2421f20 None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:40.827255 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6baabb25-96c4-4273-99b3-cd7582ef33a6 None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:40.831785 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69818e54-2e72-479d-96f0-dfef4e2870a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:40.832342 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e17b3a-09ee-433b-8d85-e96f90c7059d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:40.832597 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69818e54-2e72-479d-96f0-dfef4e2870a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:40.832880 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69818e54-2e72-479d-96f0-dfef4e2870a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:40.832987 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e17b3a-09ee-433b-8d85-e96f90c7059d admin admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:40.833143 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69818e54-2e72-479d-96f0-dfef4e2870a5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:40.833239 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e17b3a-09ee-433b-8d85-e96f90c7059d admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:40.833511 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e17b3a-09ee-433b-8d85-e96f90c7059d admin admin] PATH_INFO: `/v3/domains` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:40.833785 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69818e54-2e72-479d-96f0-dfef4e2870a5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:41:40.834086 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69818e54-2e72-479d-96f0-dfef4e2870a5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:40.834966 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e17b3a-09ee-433b-8d85-e96f90c7059d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:41:40.835340 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e17b3a-09ee-433b-8d85-e96f90c7059d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:40.835699 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69818e54-2e72-479d-96f0-dfef4e2870a5 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:40.837877 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e17b3a-09ee-433b-8d85-e96f90c7059d admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:40.852435 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1053/2112] 10.222.0.45 () {58 vars in 1153 bytes} [Wed Aug 24 17:41:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:41:40.852650 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1060/2113] 10.222.0.45 () {58 vars in 1153 bytes} [Wed Aug 24 17:41:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:41:40.853464 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69818e54-2e72-479d-96f0-dfef4e2870a5 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:40.854697 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e17b3a-09ee-433b-8d85-e96f90c7059d admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:40.865633 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69818e54-2e72-479d-96f0-dfef4e2870a5 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:40.868751 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e17b3a-09ee-433b-8d85-e96f90c7059d admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:40.870239 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d2b228-474f-4422-8f48-1c5ca1c48c9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:40.870882 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d2b228-474f-4422-8f48-1c5ca1c48c9a admin admin] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:40.871147 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d2b228-474f-4422-8f48-1c5ca1c48c9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:40.871423 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d2b228-474f-4422-8f48-1c5ca1c48c9a admin admin] PATH_INFO: `/v3/projects` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:40.872054 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d2b228-474f-4422-8f48-1c5ca1c48c9a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:40.873921 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fccd60-3806-490d-9eea-5aa3af0fbb55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:40.874173 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d2b228-474f-4422-8f48-1c5ca1c48c9a admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:40.874529 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fccd60-3806-490d-9eea-5aa3af0fbb55 admin admin] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:40.874809 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fccd60-3806-490d-9eea-5aa3af0fbb55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:40.875084 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fccd60-3806-490d-9eea-5aa3af0fbb55 admin admin] PATH_INFO: `/v3/projects` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:40.875717 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fccd60-3806-490d-9eea-5aa3af0fbb55 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:40.878251 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fccd60-3806-490d-9eea-5aa3af0fbb55 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:40.908285 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1061/2114] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 17:41:40 2022] POST /identity/v3/projects => generated 381 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:41:40.908722 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1054/2115] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 17:41:40 2022] POST /identity/v3/projects => generated 367 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:41:40.910037 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fccd60-3806-490d-9eea-5aa3af0fbb55 admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:40.912379 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d2b228-474f-4422-8f48-1c5ca1c48c9a admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:40.959137 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:41:40.975279 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fccd60-3806-490d-9eea-5aa3af0fbb55 admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:40.975988 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d2b228-474f-4422-8f48-1c5ca1c48c9a admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:40.980533 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73df093-c0f6-4970-9824-a8c91d0ac76a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:40.980977 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d60016-22f0-4ac4-95ee-d8f3194352b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:40.981253 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73df093-c0f6-4970-9824-a8c91d0ac76a admin admin] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:40.981519 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73df093-c0f6-4970-9824-a8c91d0ac76a admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:40.981623 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d60016-22f0-4ac4-95ee-d8f3194352b0 admin admin] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:40.981802 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73df093-c0f6-4970-9824-a8c91d0ac76a admin admin] PATH_INFO: `/v3/users` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:40.981901 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d60016-22f0-4ac4-95ee-d8f3194352b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:40.982077 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d60016-22f0-4ac4-95ee-d8f3194352b0 admin admin] PATH_INFO: `/v3/users` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:40.982458 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73df093-c0f6-4970-9824-a8c91d0ac76a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:40.982703 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d60016-22f0-4ac4-95ee-d8f3194352b0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:40.984386 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73df093-c0f6-4970-9824-a8c91d0ac76a admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:40.984642 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d60016-22f0-4ac4-95ee-d8f3194352b0 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:41.009637 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1055/2116] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 17:41:40 2022] POST /identity/v3/users => generated 291 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:41:41.013141 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1062/2117] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 17:41:40 2022] POST /identity/v3/users => generated 284 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:41:41.014244 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73df093-c0f6-4970-9824-a8c91d0ac76a admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:41.014356 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d60016-22f0-4ac4-95ee-d8f3194352b0 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:41.026166 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d60016-22f0-4ac4-95ee-d8f3194352b0 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:41.028824 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73df093-c0f6-4970-9824-a8c91d0ac76a admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:41.033862 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c495fb4-df14-4de9-9f37-831073dae274 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:41.034478 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c495fb4-df14-4de9-9f37-831073dae274 admin admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:41.034753 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c495fb4-df14-4de9-9f37-831073dae274 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:41.034852 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da586c3-11fe-497f-9f42-9c066ac8706e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:41.035020 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c495fb4-df14-4de9-9f37-831073dae274 admin admin] PATH_INFO: `/v3/roles` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:41.035536 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da586c3-11fe-497f-9f42-9c066ac8706e admin admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:41.035649 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c495fb4-df14-4de9-9f37-831073dae274 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:41.035887 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da586c3-11fe-497f-9f42-9c066ac8706e admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:41.036605 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da586c3-11fe-497f-9f42-9c066ac8706e admin admin] PATH_INFO: `/v3/roles` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:41.037264 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da586c3-11fe-497f-9f42-9c066ac8706e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:41.037379 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c495fb4-df14-4de9-9f37-831073dae274 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:41.039138 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da586c3-11fe-497f-9f42-9c066ac8706e admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:41.047806 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1056/2118] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:41:41 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 17:41:41.053725 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1063/2119] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:41:41 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:41:41.053852 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c495fb4-df14-4de9-9f37-831073dae274 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:41.054642 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da586c3-11fe-497f-9f42-9c066ac8706e admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:41.062330 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c495fb4-df14-4de9-9f37-831073dae274 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:41.065407 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da586c3-11fe-497f-9f42-9c066ac8706e admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:41.068621 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05fac95-25fd-46f8-9fc6-bb114fffe015 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:41.068621 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05fac95-25fd-46f8-9fc6-bb114fffe015 admin admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:41.068621 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05fac95-25fd-46f8-9fc6-bb114fffe015 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:41.068621 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05fac95-25fd-46f8-9fc6-bb114fffe015 admin admin] PATH_INFO: `/v3/roles` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:41.071366 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05fac95-25fd-46f8-9fc6-bb114fffe015 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:41.071496 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640d1fbf-8217-4859-9971-aad3e1bf1382 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:41.071588 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05fac95-25fd-46f8-9fc6-bb114fffe015 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:41.071659 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640d1fbf-8217-4859-9971-aad3e1bf1382 admin admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:41.071659 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640d1fbf-8217-4859-9971-aad3e1bf1382 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:41.071895 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640d1fbf-8217-4859-9971-aad3e1bf1382 admin admin] PATH_INFO: `/v3/roles` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:41.072837 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640d1fbf-8217-4859-9971-aad3e1bf1382 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:41.074881 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640d1fbf-8217-4859-9971-aad3e1bf1382 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:41.082182 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1057/2120] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:41:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:41:41.086294 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1064/2121] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:41:41 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 17:41:41.086417 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05fac95-25fd-46f8-9fc6-bb114fffe015 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:41.090090 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640d1fbf-8217-4859-9971-aad3e1bf1382 admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:41.097056 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05fac95-25fd-46f8-9fc6-bb114fffe015 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:41.100099 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640d1fbf-8217-4859-9971-aad3e1bf1382 admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:41.101769 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4ab8d5-815c-4801-b5f0-1dd77112171d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:41.102433 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4ab8d5-815c-4801-b5f0-1dd77112171d admin admin] REQUEST_METHOD: `PUT` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:41.102772 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4ab8d5-815c-4801-b5f0-1dd77112171d admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:41.103083 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4ab8d5-815c-4801-b5f0-1dd77112171d admin admin] PATH_INFO: `/v3/projects/747c6d4150d34bcb94a3566933bcac48/users/4cf50f8408b84d309cae374a1a627ae2/roles/02476a5b82eb4b188c18a5d706a3ef68` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:41.104988 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efaddc40-7d52-473e-afe0-fcf131854987 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:41.105624 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaddc40-7d52-473e-afe0-fcf131854987 admin admin] REQUEST_METHOD: `PUT` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:41.105905 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaddc40-7d52-473e-afe0-fcf131854987 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:41.106182 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaddc40-7d52-473e-afe0-fcf131854987 admin admin] PATH_INFO: `/v3/projects/d10a1b6bd4364e8c8c0ea743d8601461/users/a31877d157944e71b1ded9856ccd9685/roles/02476a5b82eb4b188c18a5d706a3ef68` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:41.119301 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4ab8d5-815c-4801-b5f0-1dd77112171d admin admin] RBAC: Authorizing `identity:create_grant(project_id=747c6d4150d34bcb94a3566933bcac48, user_id=4cf50f8408b84d309cae374a1a627ae2, role_id=02476a5b82eb4b188c18a5d706a3ef68)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:41.122070 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4ab8d5-815c-4801-b5f0-1dd77112171d admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:41.126647 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efaddc40-7d52-473e-afe0-fcf131854987 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d10a1b6bd4364e8c8c0ea743d8601461, user_id=a31877d157944e71b1ded9856ccd9685, role_id=02476a5b82eb4b188c18a5d706a3ef68)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:41.129305 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efaddc40-7d52-473e-afe0-fcf131854987 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:41.136170 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1058/2122] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 17:41:41 2022] PUT /identity/v3/projects/747c6d4150d34bcb94a3566933bcac48/users/4cf50f8408b84d309cae374a1a627ae2/roles/02476a5b82eb4b188c18a5d706a3ef68 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:41:41.142203 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1065/2123] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 17:41:41 2022] PUT /identity/v3/projects/d10a1b6bd4364e8c8c0ea743d8601461/users/a31877d157944e71b1ded9856ccd9685/roles/02476a5b82eb4b188c18a5d706a3ef68 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:41:41.148702 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a913e041-bc16-4dd3-a4a0-ff9944eba261 None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:41.149070 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a913e041-bc16-4dd3-a4a0-ff9944eba261 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:41.149408 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a913e041-bc16-4dd3-a4a0-ff9944eba261 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:41.152149 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f97479-6433-476d-9f53-8dd43ef6053b None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:41.152355 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f97479-6433-476d-9f53-8dd43ef6053b None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:41.152662 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f97479-6433-476d-9f53-8dd43ef6053b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:41.182182 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-a913e041-bc16-4dd3-a4a0-ff9944eba261 None None] MFA Rules not processed for user `4cf50f8408b84d309cae374a1a627ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:41:41.183386 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-a913e041-bc16-4dd3-a4a0-ff9944eba261 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:41.186635 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-75f97479-6433-476d-9f53-8dd43ef6053b None None] MFA Rules not processed for user `a31877d157944e71b1ded9856ccd9685`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:41:41.187941 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-75f97479-6433-476d-9f53-8dd43ef6053b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:41.233326 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:41:41.255612 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1059/2124] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:41:41 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:41:41.260181 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a913e041-bc16-4dd3-a4a0-ff9944eba261 None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:41.265980 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1066/2125] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:41:41 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:41:41.278462 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f97479-6433-476d-9f53-8dd43ef6053b None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:41.280405 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-75f97479-6433-476d-9f53-8dd43ef6053b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:41.329341 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a913e041-bc16-4dd3-a4a0-ff9944eba261 None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:41.333990 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802c07a6-4759-4689-8e1d-bd5ccd5ec744 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:41.334599 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802c07a6-4759-4689-8e1d-bd5ccd5ec744 admin admin] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:41.334877 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802c07a6-4759-4689-8e1d-bd5ccd5ec744 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:41.335145 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802c07a6-4759-4689-8e1d-bd5ccd5ec744 admin admin] PATH_INFO: `/v3/projects` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:41.335768 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802c07a6-4759-4689-8e1d-bd5ccd5ec744 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:41.338004 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802c07a6-4759-4689-8e1d-bd5ccd5ec744 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:41.366847 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f97479-6433-476d-9f53-8dd43ef6053b None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:41.372876 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1060/2126] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 17:41:41 2022] POST /identity/v3/projects => generated 381 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:41:41.373007 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182466d2-9577-497a-b728-1fe1e489ab90 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df5fee57539447428dd1c72eab3cfd9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:41.373007 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182466d2-9577-497a-b728-1fe1e489ab90 service glance] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:41.373007 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182466d2-9577-497a-b728-1fe1e489ab90 service glance] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:41.373007 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182466d2-9577-497a-b728-1fe1e489ab90 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:41.373957 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802c07a6-4759-4689-8e1d-bd5ccd5ec744 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:41.381537 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182466d2-9577-497a-b728-1fe1e489ab90 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:41.382984 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182466d2-9577-497a-b728-1fe1e489ab90 service glance] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:41.393133 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1067/2127] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:41:41 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 17:41:41.441777 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802c07a6-4759-4689-8e1d-bd5ccd5ec744 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:41.446498 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854e8dc9-7053-4186-be67-85c0ac3aa57d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:41.447105 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854e8dc9-7053-4186-be67-85c0ac3aa57d admin admin] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:41.447380 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854e8dc9-7053-4186-be67-85c0ac3aa57d admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:41.447649 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854e8dc9-7053-4186-be67-85c0ac3aa57d admin admin] PATH_INFO: `/v3/users` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:41.448545 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854e8dc9-7053-4186-be67-85c0ac3aa57d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:41.450611 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854e8dc9-7053-4186-be67-85c0ac3aa57d admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:41.474367 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1061/2128] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 17:41:41 2022] POST /identity/v3/users => generated 297 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:41:41.478853 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182466d2-9577-497a-b728-1fe1e489ab90 service glance] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:41.484877 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:41:41.492514 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182466d2-9577-497a-b728-1fe1e489ab90 service glance] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:41.497105 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f68a1e6-2050-4323-a789-7b07aa7c802c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:41.497707 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f68a1e6-2050-4323-a789-7b07aa7c802c admin admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:41.497982 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f68a1e6-2050-4323-a789-7b07aa7c802c admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:41.498251 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f68a1e6-2050-4323-a789-7b07aa7c802c admin admin] PATH_INFO: `/v3/roles` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:41.498953 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f68a1e6-2050-4323-a789-7b07aa7c802c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:41.500565 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f68a1e6-2050-4323-a789-7b07aa7c802c admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:41.514361 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1068/2129] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:41:41 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:41:41.515444 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854e8dc9-7053-4186-be67-85c0ac3aa57d admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:41.526265 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854e8dc9-7053-4186-be67-85c0ac3aa57d admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:41.531650 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca104c5d-b190-4834-a2b8-ed733adba327 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:41.532356 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca104c5d-b190-4834-a2b8-ed733adba327 admin admin] REQUEST_METHOD: `PUT` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:41.533549 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca104c5d-b190-4834-a2b8-ed733adba327 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:41.533549 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca104c5d-b190-4834-a2b8-ed733adba327 admin admin] PATH_INFO: `/v3/projects/72ba429a1be0410b9e9a3f9a4be6b27d/users/6e369e5697de45669ce637e5a034cae4/roles/e411ad9eb96e4e8d85b59b5b700ada7b` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:41.549283 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca104c5d-b190-4834-a2b8-ed733adba327 admin admin] RBAC: Authorizing `identity:create_grant(project_id=72ba429a1be0410b9e9a3f9a4be6b27d, user_id=6e369e5697de45669ce637e5a034cae4, role_id=e411ad9eb96e4e8d85b59b5b700ada7b)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:41.551976 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca104c5d-b190-4834-a2b8-ed733adba327 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:41.565107 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1062/2130] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 17:41:41 2022] PUT /identity/v3/projects/72ba429a1be0410b9e9a3f9a4be6b27d/users/6e369e5697de45669ce637e5a034cae4/roles/e411ad9eb96e4e8d85b59b5b700ada7b => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:41:41.568945 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f68a1e6-2050-4323-a789-7b07aa7c802c admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:41.579461 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f68a1e6-2050-4323-a789-7b07aa7c802c admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:41.637888 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472c1071-998d-4800-98e3-6a9ec7e9c051 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:41.639096 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472c1071-998d-4800-98e3-6a9ec7e9c051 admin admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:41.639096 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472c1071-998d-4800-98e3-6a9ec7e9c051 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:41.639287 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472c1071-998d-4800-98e3-6a9ec7e9c051 admin admin] PATH_INFO: `/v3/roles` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:41.639924 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472c1071-998d-4800-98e3-6a9ec7e9c051 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:41.641773 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472c1071-998d-4800-98e3-6a9ec7e9c051 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:41.656217 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1069/2131] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:41:41 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:41:41.657591 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca104c5d-b190-4834-a2b8-ed733adba327 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:41.668234 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca104c5d-b190-4834-a2b8-ed733adba327 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:41.673229 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5230e8d9-4fe6-4bbf-aefc-aa8a80362356 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:41.673897 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5230e8d9-4fe6-4bbf-aefc-aa8a80362356 admin admin] REQUEST_METHOD: `PUT` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:41.674173 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5230e8d9-4fe6-4bbf-aefc-aa8a80362356 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:41.674442 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5230e8d9-4fe6-4bbf-aefc-aa8a80362356 admin admin] PATH_INFO: `/v3/projects/72ba429a1be0410b9e9a3f9a4be6b27d/users/6e369e5697de45669ce637e5a034cae4/roles/e411ad9eb96e4e8d85b59b5b700ada7b` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:41.676314 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5230e8d9-4fe6-4bbf-aefc-aa8a80362356 admin admin] RBAC: Authorizing `identity:create_grant(project_id=72ba429a1be0410b9e9a3f9a4be6b27d, user_id=6e369e5697de45669ce637e5a034cae4, role_id=e411ad9eb96e4e8d85b59b5b700ada7b)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:41.679044 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5230e8d9-4fe6-4bbf-aefc-aa8a80362356 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:41.690774 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1063/2132] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 17:41:41 2022] PUT /identity/v3/projects/72ba429a1be0410b9e9a3f9a4be6b27d/users/6e369e5697de45669ce637e5a034cae4/roles/e411ad9eb96e4e8d85b59b5b700ada7b => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:41:41.700338 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bec5680-77fb-4e79-af9c-cadb5fc62fb1 None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:41.700660 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bec5680-77fb-4e79-af9c-cadb5fc62fb1 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:41.700924 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bec5680-77fb-4e79-af9c-cadb5fc62fb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:41.732744 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-8bec5680-77fb-4e79-af9c-cadb5fc62fb1 None None] MFA Rules not processed for user `6e369e5697de45669ce637e5a034cae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:41:41.733942 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-8bec5680-77fb-4e79-af9c-cadb5fc62fb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:41.831616 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1070/2133] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:41:41 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:41:41.844681 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5230e8d9-4fe6-4bbf-aefc-aa8a80362356 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:41.847278 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-5230e8d9-4fe6-4bbf-aefc-aa8a80362356 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:41.877431 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:41:41.910775 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5230e8d9-4fe6-4bbf-aefc-aa8a80362356 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:41.914342 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c12ef66-c99e-4e5f-aeb9-4807000cd14c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59be342541264942ad87f60ff4de6c8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:41.914953 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c12ef66-c99e-4e5f-aeb9-4807000cd14c service cinder] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:41.915234 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c12ef66-c99e-4e5f-aeb9-4807000cd14c service cinder] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:41.915509 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c12ef66-c99e-4e5f-aeb9-4807000cd14c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:41.924875 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c12ef66-c99e-4e5f-aeb9-4807000cd14c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:41.926320 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c12ef66-c99e-4e5f-aeb9-4807000cd14c service cinder] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:41.937179 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1064/2134] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:41:41 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:41:42.321850 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bec5680-77fb-4e79-af9c-cadb5fc62fb1 None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:42.383313 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bec5680-77fb-4e79-af9c-cadb5fc62fb1 None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:42.388082 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02fad52-65f7-4a27-895d-85db267f7834 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:42.388988 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02fad52-65f7-4a27-895d-85db267f7834 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:42.389279 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02fad52-65f7-4a27-895d-85db267f7834 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:42.389554 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02fad52-65f7-4a27-895d-85db267f7834 admin admin] PATH_INFO: `/v3/users/4cf50f8408b84d309cae374a1a627ae2` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:42.390682 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02fad52-65f7-4a27-895d-85db267f7834 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4cf50f8408b84d309cae374a1a627ae2)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:42.392727 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02fad52-65f7-4a27-895d-85db267f7834 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:42.406198 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:41:42.438512 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-c02fad52-65f7-4a27-895d-85db267f7834 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4cf50f8408b84d309cae374a1a627ae2'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:42.439171 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-c02fad52-65f7-4a27-895d-85db267f7834 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4cf50f8408b84d309cae374a1a627ae2'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:42.446683 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-c02fad52-65f7-4a27-895d-85db267f7834 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4cf50f8408b84d309cae374a1a627ae2'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:42.447262 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-c02fad52-65f7-4a27-895d-85db267f7834 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4cf50f8408b84d309cae374a1a627ae2'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:42.458889 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-c02fad52-65f7-4a27-895d-85db267f7834 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4cf50f8408b84d309cae374a1a627ae2'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:42.470579 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1071/2135] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 17:41:42 2022] DELETE /identity/v3/users/4cf50f8408b84d309cae374a1a627ae2 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:41:42.475032 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a9f0e5-128f-414a-8832-d1dc70719b8f None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:42.475317 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a9f0e5-128f-414a-8832-d1dc70719b8f None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:42.475618 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a9f0e5-128f-414a-8832-d1dc70719b8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:42.494737 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-98a9f0e5-128f-414a-8832-d1dc70719b8f None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:41:42.495950 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-98a9f0e5-128f-414a-8832-d1dc70719b8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:42.562388 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1065/2136] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:41:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:41:42.571960 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02fad52-65f7-4a27-895d-85db267f7834 admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:42.573400 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-c02fad52-65f7-4a27-895d-85db267f7834 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:42.633184 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02fad52-65f7-4a27-895d-85db267f7834 admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:42.636797 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a500f38-063a-4555-a81c-ac751d8ad391 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43ad98a250a144099ccd18378aedd4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:42.637404 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a500f38-063a-4555-a81c-ac751d8ad391 service neutron] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:42.637679 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a500f38-063a-4555-a81c-ac751d8ad391 service neutron] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:42.637957 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a500f38-063a-4555-a81c-ac751d8ad391 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:42.647942 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a500f38-063a-4555-a81c-ac751d8ad391 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:42.649395 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a500f38-063a-4555-a81c-ac751d8ad391 service neutron] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:42.659581 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1072/2137] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:41:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:41:42.918704 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a9f0e5-128f-414a-8832-d1dc70719b8f None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:42.920500 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-98a9f0e5-128f-414a-8832-d1dc70719b8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:42.933419 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a9f0e5-128f-414a-8832-d1dc70719b8f None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:42.938761 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c966d3-4095-4a9b-badd-6bd48152aa6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:42.939686 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c966d3-4095-4a9b-badd-6bd48152aa6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:42.939795 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c966d3-4095-4a9b-badd-6bd48152aa6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:42.940065 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c966d3-4095-4a9b-badd-6bd48152aa6d admin admin] PATH_INFO: `/v3/projects/747c6d4150d34bcb94a3566933bcac48` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:42.944757 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c966d3-4095-4a9b-badd-6bd48152aa6d admin admin] RBAC: Authorizing `identity:delete_project(project_id=747c6d4150d34bcb94a3566933bcac48)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:42.946758 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c966d3-4095-4a9b-badd-6bd48152aa6d admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:42.994473 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-00c966d3-4095-4a9b-badd-6bd48152aa6d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '747c6d4150d34bcb94a3566933bcac48', 'request_id': 'req-00c966d3-4095-4a9b-badd-6bd48152aa6d'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:42.999892 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-00c966d3-4095-4a9b-badd-6bd48152aa6d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '747c6d4150d34bcb94a3566933bcac48', 'request_id': 'req-00c966d3-4095-4a9b-badd-6bd48152aa6d'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:43.005814 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-00c966d3-4095-4a9b-badd-6bd48152aa6d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '747c6d4150d34bcb94a3566933bcac48', 'request_id': 'req-00c966d3-4095-4a9b-badd-6bd48152aa6d'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:43.018309 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-00c966d3-4095-4a9b-badd-6bd48152aa6d admin admin] The token cache is being invalidate because project 747c6d4150d34bcb94a3566933bcac48 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81759) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:41:43.018593 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-00c966d3-4095-4a9b-badd-6bd48152aa6d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:43.019196 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-00c966d3-4095-4a9b-badd-6bd48152aa6d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:43.020665 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1066/2138] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 17:41:42 2022] DELETE /identity/v3/projects/747c6d4150d34bcb94a3566933bcac48 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:41:43.024647 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a500f38-063a-4555-a81c-ac751d8ad391 service neutron] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:43.034880 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a500f38-063a-4555-a81c-ac751d8ad391 service neutron] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:43.088863 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409b226e-0d83-4d00-8c89-82ca00010968 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:43.089490 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409b226e-0d83-4d00-8c89-82ca00010968 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:43.089769 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409b226e-0d83-4d00-8c89-82ca00010968 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:43.090045 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409b226e-0d83-4d00-8c89-82ca00010968 admin admin] PATH_INFO: `/v3/users/6e369e5697de45669ce637e5a034cae4` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:43.091155 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409b226e-0d83-4d00-8c89-82ca00010968 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e369e5697de45669ce637e5a034cae4)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:43.093100 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409b226e-0d83-4d00-8c89-82ca00010968 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:43.134831 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:41:43.145934 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-409b226e-0d83-4d00-8c89-82ca00010968 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e369e5697de45669ce637e5a034cae4'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:43.146582 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-409b226e-0d83-4d00-8c89-82ca00010968 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e369e5697de45669ce637e5a034cae4'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:43.154910 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-409b226e-0d83-4d00-8c89-82ca00010968 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e369e5697de45669ce637e5a034cae4'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:43.155363 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-409b226e-0d83-4d00-8c89-82ca00010968 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e369e5697de45669ce637e5a034cae4'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:43.170077 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-409b226e-0d83-4d00-8c89-82ca00010968 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e369e5697de45669ce637e5a034cae4'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:43.182439 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1073/2139] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 17:41:43 2022] DELETE /identity/v3/users/6e369e5697de45669ce637e5a034cae4 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:41:43.456600 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c966d3-4095-4a9b-badd-6bd48152aa6d admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:43.458053 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-00c966d3-4095-4a9b-badd-6bd48152aa6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:43.472970 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c966d3-4095-4a9b-badd-6bd48152aa6d admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:43.521174 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:41:43.532114 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc06b058-cf91-46f8-a9f7-4cfca2bd682d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:43.532726 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc06b058-cf91-46f8-a9f7-4cfca2bd682d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:43.533027 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc06b058-cf91-46f8-a9f7-4cfca2bd682d admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:43.533297 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc06b058-cf91-46f8-a9f7-4cfca2bd682d admin admin] PATH_INFO: `/v3/projects/72ba429a1be0410b9e9a3f9a4be6b27d` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:43.534394 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc06b058-cf91-46f8-a9f7-4cfca2bd682d admin admin] RBAC: Authorizing `identity:delete_project(project_id=72ba429a1be0410b9e9a3f9a4be6b27d)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:43.536295 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc06b058-cf91-46f8-a9f7-4cfca2bd682d admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:43.581686 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-bc06b058-cf91-46f8-a9f7-4cfca2bd682d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '72ba429a1be0410b9e9a3f9a4be6b27d', 'request_id': 'req-bc06b058-cf91-46f8-a9f7-4cfca2bd682d'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:43.587131 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-bc06b058-cf91-46f8-a9f7-4cfca2bd682d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '72ba429a1be0410b9e9a3f9a4be6b27d', 'request_id': 'req-bc06b058-cf91-46f8-a9f7-4cfca2bd682d'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:43.593144 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-bc06b058-cf91-46f8-a9f7-4cfca2bd682d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '72ba429a1be0410b9e9a3f9a4be6b27d', 'request_id': 'req-bc06b058-cf91-46f8-a9f7-4cfca2bd682d'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:43.604878 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-bc06b058-cf91-46f8-a9f7-4cfca2bd682d admin admin] The token cache is being invalidate because project 72ba429a1be0410b9e9a3f9a4be6b27d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81759) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:41:43.605164 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-bc06b058-cf91-46f8-a9f7-4cfca2bd682d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:43.605776 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-bc06b058-cf91-46f8-a9f7-4cfca2bd682d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:43.607228 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1067/2140] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 17:41:43 2022] DELETE /identity/v3/projects/72ba429a1be0410b9e9a3f9a4be6b27d => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:41:43.612384 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a74528-1eb9-4199-8dc1-717dfef73c14 None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:43.612721 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a74528-1eb9-4199-8dc1-717dfef73c14 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:43.613029 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a74528-1eb9-4199-8dc1-717dfef73c14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:43.632616 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-e6a74528-1eb9-4199-8dc1-717dfef73c14 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:41:43.633851 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-e6a74528-1eb9-4199-8dc1-717dfef73c14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:43.693640 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1074/2141] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:41:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:41:43.701470 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02910102-3690-43a5-9730-b14f9c41f75b None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:43.701754 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02910102-3690-43a5-9730-b14f9c41f75b None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:43.702028 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02910102-3690-43a5-9730-b14f9c41f75b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:43.721770 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-02910102-3690-43a5-9730-b14f9c41f75b None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:41:43.722952 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-02910102-3690-43a5-9730-b14f9c41f75b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:43.730721 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1068/2142] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:41:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:41:43.734943 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a74528-1eb9-4199-8dc1-717dfef73c14 None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:43.745034 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a74528-1eb9-4199-8dc1-717dfef73c14 None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:43.749612 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2135e87e-3bab-4066-ac69-6ba28195f71f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:43.750202 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2135e87e-3bab-4066-ac69-6ba28195f71f admin admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:43.750481 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2135e87e-3bab-4066-ac69-6ba28195f71f admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:43.750752 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2135e87e-3bab-4066-ac69-6ba28195f71f admin admin] PATH_INFO: `/v3/domains` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:43.751392 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2135e87e-3bab-4066-ac69-6ba28195f71f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:41:43.751694 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2135e87e-3bab-4066-ac69-6ba28195f71f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:43.754015 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2135e87e-3bab-4066-ac69-6ba28195f71f admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:43.765676 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1075/2143] 10.222.0.45 () {58 vars in 1153 bytes} [Wed Aug 24 17:41:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:41:43.770448 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02910102-3690-43a5-9730-b14f9c41f75b None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:43.779104 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02910102-3690-43a5-9730-b14f9c41f75b None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:43.783864 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411ece01-9b66-462a-83ec-17a55a1291e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:43.784463 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411ece01-9b66-462a-83ec-17a55a1291e3 admin admin] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:43.784805 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411ece01-9b66-462a-83ec-17a55a1291e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:43.785075 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411ece01-9b66-462a-83ec-17a55a1291e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:43.785756 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411ece01-9b66-462a-83ec-17a55a1291e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:43.787682 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411ece01-9b66-462a-83ec-17a55a1291e3 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:43.815162 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1069/2144] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 17:41:43 2022] POST /identity/v3/projects => generated 385 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:41:43.819398 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2135e87e-3bab-4066-ac69-6ba28195f71f admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:43.829559 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2135e87e-3bab-4066-ac69-6ba28195f71f admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:43.885026 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975706fc-3054-4f47-aa89-6bc89554c078 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:43.885620 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975706fc-3054-4f47-aa89-6bc89554c078 admin admin] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:43.885897 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975706fc-3054-4f47-aa89-6bc89554c078 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:43.886166 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975706fc-3054-4f47-aa89-6bc89554c078 admin admin] PATH_INFO: `/v3/users` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:43.886798 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975706fc-3054-4f47-aa89-6bc89554c078 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:43.889331 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975706fc-3054-4f47-aa89-6bc89554c078 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:43.909333 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1076/2145] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 17:41:43 2022] POST /identity/v3/users => generated 293 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:41:43.913516 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411ece01-9b66-462a-83ec-17a55a1291e3 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:43.924186 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411ece01-9b66-462a-83ec-17a55a1291e3 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:43.929480 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60eb4cd6-ab36-4569-9dc3-da0f8296e0f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:43.930087 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60eb4cd6-ab36-4569-9dc3-da0f8296e0f6 admin admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:43.930367 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60eb4cd6-ab36-4569-9dc3-da0f8296e0f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:43.930642 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60eb4cd6-ab36-4569-9dc3-da0f8296e0f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:43.931288 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60eb4cd6-ab36-4569-9dc3-da0f8296e0f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:43.932920 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60eb4cd6-ab36-4569-9dc3-da0f8296e0f6 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:43.944096 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1070/2146] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:41:43 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:41:43.948064 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975706fc-3054-4f47-aa89-6bc89554c078 admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:43.958129 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975706fc-3054-4f47-aa89-6bc89554c078 admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:43.963687 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff83aba-8843-4934-9afb-2ffad637eb18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:43.964288 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff83aba-8843-4934-9afb-2ffad637eb18 admin admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:43.964597 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff83aba-8843-4934-9afb-2ffad637eb18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:43.964872 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff83aba-8843-4934-9afb-2ffad637eb18 admin admin] PATH_INFO: `/v3/roles` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:43.965516 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff83aba-8843-4934-9afb-2ffad637eb18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:43.967103 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff83aba-8843-4934-9afb-2ffad637eb18 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:43.978394 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1077/2147] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:41:43 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:41:43.982612 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60eb4cd6-ab36-4569-9dc3-da0f8296e0f6 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:43.993140 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60eb4cd6-ab36-4569-9dc3-da0f8296e0f6 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:43.997729 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa9fee3-eae4-4d26-a6bb-7956c0f8ad73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:43.998377 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa9fee3-eae4-4d26-a6bb-7956c0f8ad73 admin admin] REQUEST_METHOD: `PUT` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:43.998658 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa9fee3-eae4-4d26-a6bb-7956c0f8ad73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:43.998936 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa9fee3-eae4-4d26-a6bb-7956c0f8ad73 admin admin] PATH_INFO: `/v3/projects/e11108fa70134dd79435bfcc31f0b40c/users/72fdc5929f604135881d4984cf01c4cd/roles/02476a5b82eb4b188c18a5d706a3ef68` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:44.014440 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa9fee3-eae4-4d26-a6bb-7956c0f8ad73 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e11108fa70134dd79435bfcc31f0b40c, user_id=72fdc5929f604135881d4984cf01c4cd, role_id=02476a5b82eb4b188c18a5d706a3ef68)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:44.017204 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa9fee3-eae4-4d26-a6bb-7956c0f8ad73 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:44.029883 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1071/2148] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 17:41:43 2022] PUT /identity/v3/projects/e11108fa70134dd79435bfcc31f0b40c/users/72fdc5929f604135881d4984cf01c4cd/roles/02476a5b82eb4b188c18a5d706a3ef68 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:41:44.041451 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26122c00-72d4-4c49-917f-ee616f3683e6 None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:44.041916 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26122c00-72d4-4c49-917f-ee616f3683e6 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:44.042015 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26122c00-72d4-4c49-917f-ee616f3683e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:44.069240 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-26122c00-72d4-4c49-917f-ee616f3683e6 None None] MFA Rules not processed for user `72fdc5929f604135881d4984cf01c4cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:41:44.070440 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-26122c00-72d4-4c49-917f-ee616f3683e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:44.078674 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:41:44.136826 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1078/2149] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:41:44 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:41:44.147906 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa9fee3-eae4-4d26-a6bb-7956c0f8ad73 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:44.149649 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-daa9fee3-eae4-4d26-a6bb-7956c0f8ad73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:44.208280 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa9fee3-eae4-4d26-a6bb-7956c0f8ad73 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:44.211932 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e2e912-09d7-4815-95f5-eaf189a1611a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59be342541264942ad87f60ff4de6c8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:44.212536 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e2e912-09d7-4815-95f5-eaf189a1611a service cinder] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:44.212844 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e2e912-09d7-4815-95f5-eaf189a1611a service cinder] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:44.213163 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e2e912-09d7-4815-95f5-eaf189a1611a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:44.221810 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e2e912-09d7-4815-95f5-eaf189a1611a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:44.223233 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e2e912-09d7-4815-95f5-eaf189a1611a service cinder] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:44.231894 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1072/2150] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:41:44 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:41:54.134010 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=23 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:41:54.227930 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=81,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:42:01.134998 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26122c00-72d4-4c49-917f-ee616f3683e6 None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:42:01.137024 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e2e912-09d7-4815-95f5-eaf189a1611a service cinder] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:42:01.137415 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-26122c00-72d4-4c49-917f-ee616f3683e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:01.220183 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e2e912-09d7-4815-95f5-eaf189a1611a service cinder] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:42:01.225455 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656b22a6-3e74-478d-bd41-97ce55a511c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:42:01.226093 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656b22a6-3e74-478d-bd41-97ce55a511c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:01.226507 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656b22a6-3e74-478d-bd41-97ce55a511c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:01.226810 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656b22a6-3e74-478d-bd41-97ce55a511c0 admin admin] PATH_INFO: `/v3/users/72fdc5929f604135881d4984cf01c4cd` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:01.227941 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656b22a6-3e74-478d-bd41-97ce55a511c0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=72fdc5929f604135881d4984cf01c4cd)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:42:01.230237 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656b22a6-3e74-478d-bd41-97ce55a511c0 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:42:01.231586 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26122c00-72d4-4c49-917f-ee616f3683e6 None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:42:01.236500 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dee9080-87ff-46cf-9b30-3a2ae029e8d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:42:01.237449 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dee9080-87ff-46cf-9b30-3a2ae029e8d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:01.237724 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dee9080-87ff-46cf-9b30-3a2ae029e8d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:01.237994 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dee9080-87ff-46cf-9b30-3a2ae029e8d8 admin admin] PATH_INFO: `/v3/users/8460d762680649339dc1d5398a9094ee` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:01.239172 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dee9080-87ff-46cf-9b30-3a2ae029e8d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8460d762680649339dc1d5398a9094ee)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:42:01.241865 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dee9080-87ff-46cf-9b30-3a2ae029e8d8 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:42:01.288785 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-656b22a6-3e74-478d-bd41-97ce55a511c0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '72fdc5929f604135881d4984cf01c4cd'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:42:01.288914 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-656b22a6-3e74-478d-bd41-97ce55a511c0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '72fdc5929f604135881d4984cf01c4cd'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:42:01.297602 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-656b22a6-3e74-478d-bd41-97ce55a511c0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '72fdc5929f604135881d4984cf01c4cd'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:42:01.298324 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-656b22a6-3e74-478d-bd41-97ce55a511c0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '72fdc5929f604135881d4984cf01c4cd'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:42:01.302372 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-1dee9080-87ff-46cf-9b30-3a2ae029e8d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8460d762680649339dc1d5398a9094ee'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:42:01.303404 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-1dee9080-87ff-46cf-9b30-3a2ae029e8d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8460d762680649339dc1d5398a9094ee'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:42:01.306751 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-656b22a6-3e74-478d-bd41-97ce55a511c0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '72fdc5929f604135881d4984cf01c4cd'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:42:01.311642 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-1dee9080-87ff-46cf-9b30-3a2ae029e8d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8460d762680649339dc1d5398a9094ee'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:42:01.312378 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-1dee9080-87ff-46cf-9b30-3a2ae029e8d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8460d762680649339dc1d5398a9094ee'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:42:01.326551 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-1dee9080-87ff-46cf-9b30-3a2ae029e8d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8460d762680649339dc1d5398a9094ee'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:42:01.328179 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1073/2151] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 17:42:01 2022] DELETE /identity/v3/users/72fdc5929f604135881d4984cf01c4cd => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:42:01.332602 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01945705-b698-4258-9f3e-111045f62c0a None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:01.332999 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01945705-b698-4258-9f3e-111045f62c0a None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:01.333678 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01945705-b698-4258-9f3e-111045f62c0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:01.341050 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1079/2152] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 17:42:01 2022] DELETE /identity/v3/users/8460d762680649339dc1d5398a9094ee => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:42:01.345073 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71065c7f-0e05-46d1-96e7-a04368e27c2a None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:01.345466 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71065c7f-0e05-46d1-96e7-a04368e27c2a None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:01.345841 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71065c7f-0e05-46d1-96e7-a04368e27c2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:01.354761 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-01945705-b698-4258-9f3e-111045f62c0a None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:42:01.355835 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-01945705-b698-4258-9f3e-111045f62c0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:01.365748 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-71065c7f-0e05-46d1-96e7-a04368e27c2a None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:42:01.367060 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-71065c7f-0e05-46d1-96e7-a04368e27c2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:01.424833 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1074/2153] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:42:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:42:01.431193 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1080/2154] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:42:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:42:01.434992 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01945705-b698-4258-9f3e-111045f62c0a None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:42:01.436333 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-01945705-b698-4258-9f3e-111045f62c0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:01.440399 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71065c7f-0e05-46d1-96e7-a04368e27c2a None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:42:01.441754 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-71065c7f-0e05-46d1-96e7-a04368e27c2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:01.460007 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:42:01.462977 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:42:01.509664 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01945705-b698-4258-9f3e-111045f62c0a None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:42:01.515497 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35c670e-b5ef-4a4f-b171-b2cddb5c7e32 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43ad98a250a144099ccd18378aedd4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:42:01.516186 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35c670e-b5ef-4a4f-b171-b2cddb5c7e32 service neutron] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:01.516607 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35c670e-b5ef-4a4f-b171-b2cddb5c7e32 service neutron] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:01.516960 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35c670e-b5ef-4a4f-b171-b2cddb5c7e32 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:01.521110 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71065c7f-0e05-46d1-96e7-a04368e27c2a None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:42:01.524755 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ffb5f8-0124-40b2-ae44-ce879214b772 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43ad98a250a144099ccd18378aedd4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:42:01.525442 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ffb5f8-0124-40b2-ae44-ce879214b772 service neutron] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:01.525718 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ffb5f8-0124-40b2-ae44-ce879214b772 service neutron] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:01.526016 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ffb5f8-0124-40b2-ae44-ce879214b772 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:01.527252 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35c670e-b5ef-4a4f-b171-b2cddb5c7e32 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:42:01.528725 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35c670e-b5ef-4a4f-b171-b2cddb5c7e32 service neutron] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:42:01.536939 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ffb5f8-0124-40b2-ae44-ce879214b772 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:42:01.538518 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ffb5f8-0124-40b2-ae44-ce879214b772 service neutron] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:42:01.539445 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1075/2155] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:42:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 17:42:01.549350 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1081/2156] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:42:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:42:01.940344 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35c670e-b5ef-4a4f-b171-b2cddb5c7e32 service neutron] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:42:01.941916 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-e35c670e-b5ef-4a4f-b171-b2cddb5c7e32 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:01.956011 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35c670e-b5ef-4a4f-b171-b2cddb5c7e32 service neutron] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:42:01.961114 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790eeb4c-9f2b-4e51-ac3f-4febdf7f03a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:42:01.961889 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790eeb4c-9f2b-4e51-ac3f-4febdf7f03a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:01.962273 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790eeb4c-9f2b-4e51-ac3f-4febdf7f03a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:01.962649 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790eeb4c-9f2b-4e51-ac3f-4febdf7f03a4 admin admin] PATH_INFO: `/v3/projects/3d8c7e71553e4b53b71e9043cec31d4b` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:01.963941 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790eeb4c-9f2b-4e51-ac3f-4febdf7f03a4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3d8c7e71553e4b53b71e9043cec31d4b)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:42:01.965993 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790eeb4c-9f2b-4e51-ac3f-4febdf7f03a4 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:42:02.018355 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-790eeb4c-9f2b-4e51-ac3f-4febdf7f03a4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3d8c7e71553e4b53b71e9043cec31d4b', 'request_id': 'req-790eeb4c-9f2b-4e51-ac3f-4febdf7f03a4'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:42:02.024935 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-790eeb4c-9f2b-4e51-ac3f-4febdf7f03a4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3d8c7e71553e4b53b71e9043cec31d4b', 'request_id': 'req-790eeb4c-9f2b-4e51-ac3f-4febdf7f03a4'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:42:02.031650 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-790eeb4c-9f2b-4e51-ac3f-4febdf7f03a4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3d8c7e71553e4b53b71e9043cec31d4b', 'request_id': 'req-790eeb4c-9f2b-4e51-ac3f-4febdf7f03a4'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:42:02.047160 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-790eeb4c-9f2b-4e51-ac3f-4febdf7f03a4 admin admin] The token cache is being invalidate because project 3d8c7e71553e4b53b71e9043cec31d4b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81759) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:42:02.047980 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-790eeb4c-9f2b-4e51-ac3f-4febdf7f03a4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:42:02.048205 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-790eeb4c-9f2b-4e51-ac3f-4febdf7f03a4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:42:02.050067 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1076/2157] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 17:42:01 2022] DELETE /identity/v3/projects/3d8c7e71553e4b53b71e9043cec31d4b => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:42:02.054776 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad09f2e0-5170-49da-a1c2-569ff47b0d09 None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:02.055067 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad09f2e0-5170-49da-a1c2-569ff47b0d09 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:02.055347 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad09f2e0-5170-49da-a1c2-569ff47b0d09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:02.077030 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790eeb4c-9f2b-4e51-ac3f-4febdf7f03a4 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:42:02.078698 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-790eeb4c-9f2b-4e51-ac3f-4febdf7f03a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:02.083200 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-ad09f2e0-5170-49da-a1c2-569ff47b0d09 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:42:02.084461 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-ad09f2e0-5170-49da-a1c2-569ff47b0d09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:02.169451 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1082/2158] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:42:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:42:02.177834 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d80b5ba-439b-4288-91f5-7f05e4de135e None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:02.178124 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d80b5ba-439b-4288-91f5-7f05e4de135e None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:02.178795 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d80b5ba-439b-4288-91f5-7f05e4de135e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:02.191227 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790eeb4c-9f2b-4e51-ac3f-4febdf7f03a4 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:42:02.199668 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f663530-71d2-4192-b107-11ff7ead155e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:42:02.200507 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f663530-71d2-4192-b107-11ff7ead155e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:02.200887 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f663530-71d2-4192-b107-11ff7ead155e admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:02.201221 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f663530-71d2-4192-b107-11ff7ead155e admin admin] PATH_INFO: `/v3/projects/e11108fa70134dd79435bfcc31f0b40c` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:02.202315 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f663530-71d2-4192-b107-11ff7ead155e admin admin] RBAC: Authorizing `identity:delete_project(project_id=e11108fa70134dd79435bfcc31f0b40c)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:42:02.204429 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f663530-71d2-4192-b107-11ff7ead155e admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:42:02.220944 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-4d80b5ba-439b-4288-91f5-7f05e4de135e None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:42:02.222230 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-4d80b5ba-439b-4288-91f5-7f05e4de135e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:02.240264 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1083/2159] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:42:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:42:02.242150 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d80b5ba-439b-4288-91f5-7f05e4de135e None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:42:02.257249 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d80b5ba-439b-4288-91f5-7f05e4de135e None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:42:02.270008 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66a7470-2c67-4849-ae9c-4e59089cc6eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:42:02.271035 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66a7470-2c67-4849-ae9c-4e59089cc6eb admin admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:02.271695 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66a7470-2c67-4849-ae9c-4e59089cc6eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:02.273388 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66a7470-2c67-4849-ae9c-4e59089cc6eb admin admin] PATH_INFO: `/v3/domains` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:02.274075 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66a7470-2c67-4849-ae9c-4e59089cc6eb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:42:02.275817 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66a7470-2c67-4849-ae9c-4e59089cc6eb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:42:02.280674 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66a7470-2c67-4849-ae9c-4e59089cc6eb admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:42:02.299776 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:42:02.301454 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1084/2160] 10.222.0.45 () {58 vars in 1153 bytes} [Wed Aug 24 17:42:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:42:02.301824 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-5f663530-71d2-4192-b107-11ff7ead155e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e11108fa70134dd79435bfcc31f0b40c', 'request_id': 'req-5f663530-71d2-4192-b107-11ff7ead155e'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:42:02.306217 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66a7470-2c67-4849-ae9c-4e59089cc6eb admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:42:02.310234 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-5f663530-71d2-4192-b107-11ff7ead155e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e11108fa70134dd79435bfcc31f0b40c', 'request_id': 'req-5f663530-71d2-4192-b107-11ff7ead155e'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:42:02.316943 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66a7470-2c67-4849-ae9c-4e59089cc6eb admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:42:02.318235 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-5f663530-71d2-4192-b107-11ff7ead155e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e11108fa70134dd79435bfcc31f0b40c', 'request_id': 'req-5f663530-71d2-4192-b107-11ff7ead155e'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:42:02.334361 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-5f663530-71d2-4192-b107-11ff7ead155e admin admin] The token cache is being invalidate because project e11108fa70134dd79435bfcc31f0b40c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81759) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:42:02.334681 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-5f663530-71d2-4192-b107-11ff7ead155e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:42:02.335316 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-5f663530-71d2-4192-b107-11ff7ead155e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:42:02.336953 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1077/2161] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 17:42:02 2022] DELETE /identity/v3/projects/e11108fa70134dd79435bfcc31f0b40c => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:42:02.342251 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68795086-718b-4fc2-9ecd-4dc38a25428f None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:02.342530 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68795086-718b-4fc2-9ecd-4dc38a25428f None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:02.342801 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68795086-718b-4fc2-9ecd-4dc38a25428f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:02.364235 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-68795086-718b-4fc2-9ecd-4dc38a25428f None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:42:02.365474 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-68795086-718b-4fc2-9ecd-4dc38a25428f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:02.388264 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12062e80-f0eb-49b4-b2c1-925ce2943e33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:42:02.388846 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12062e80-f0eb-49b4-b2c1-925ce2943e33 admin admin] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:02.389131 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12062e80-f0eb-49b4-b2c1-925ce2943e33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:02.389407 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12062e80-f0eb-49b4-b2c1-925ce2943e33 admin admin] PATH_INFO: `/v3/projects` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:02.390301 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12062e80-f0eb-49b4-b2c1-925ce2943e33 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:42:02.392277 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12062e80-f0eb-49b4-b2c1-925ce2943e33 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:42:02.427746 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1085/2162] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 17:42:02 2022] POST /identity/v3/projects => generated 373 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:42:02.429041 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12062e80-f0eb-49b4-b2c1-925ce2943e33 admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:42:02.430986 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-12062e80-f0eb-49b4-b2c1-925ce2943e33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:02.436693 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1078/2163] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:42:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:42:02.442931 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9158408-9f87-4e21-9263-580d393cbbfa None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:02.443344 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9158408-9f87-4e21-9263-580d393cbbfa None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:02.443717 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9158408-9f87-4e21-9263-580d393cbbfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:02.494991 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-b9158408-9f87-4e21-9263-580d393cbbfa None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:42:02.496630 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-b9158408-9f87-4e21-9263-580d393cbbfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:02.501822 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:42:02.546629 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12062e80-f0eb-49b4-b2c1-925ce2943e33 admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:42:02.551716 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a14001a-bd33-4406-b6d8-5dc22dc46a3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:42:02.552325 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a14001a-bd33-4406-b6d8-5dc22dc46a3a admin admin] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:02.552640 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a14001a-bd33-4406-b6d8-5dc22dc46a3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:02.552917 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a14001a-bd33-4406-b6d8-5dc22dc46a3a admin admin] PATH_INFO: `/v3/users` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:02.553549 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a14001a-bd33-4406-b6d8-5dc22dc46a3a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:42:02.555479 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a14001a-bd33-4406-b6d8-5dc22dc46a3a admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:42:02.596097 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1086/2164] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 17:42:02 2022] POST /identity/v3/users => generated 287 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:42:02.600981 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a14001a-bd33-4406-b6d8-5dc22dc46a3a admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:42:02.601427 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1079/2165] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:42:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:42:02.602912 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9158408-9f87-4e21-9263-580d393cbbfa None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:42:02.615101 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9158408-9f87-4e21-9263-580d393cbbfa None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:42:02.620991 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a14001a-bd33-4406-b6d8-5dc22dc46a3a admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:42:02.625958 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1103ff-ead7-48c0-8813-ab4b43f42ab4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:42:02.627009 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c31f7b-3042-4769-9e04-3b545c1dbb49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:42:02.627749 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c31f7b-3042-4769-9e04-3b545c1dbb49 admin admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:02.628185 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c31f7b-3042-4769-9e04-3b545c1dbb49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:02.628494 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1103ff-ead7-48c0-8813-ab4b43f42ab4 admin admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:02.628655 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c31f7b-3042-4769-9e04-3b545c1dbb49 admin admin] PATH_INFO: `/v3/roles` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:02.629112 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1103ff-ead7-48c0-8813-ab4b43f42ab4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:02.629566 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c31f7b-3042-4769-9e04-3b545c1dbb49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:42:02.629728 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1103ff-ead7-48c0-8813-ab4b43f42ab4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:02.630379 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1103ff-ead7-48c0-8813-ab4b43f42ab4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:42:02.630695 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1103ff-ead7-48c0-8813-ab4b43f42ab4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:42:02.632195 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c31f7b-3042-4769-9e04-3b545c1dbb49 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:42:02.632324 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1103ff-ead7-48c0-8813-ab4b43f42ab4 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:42:02.649083 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1087/2166] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:42:02 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:42:02.660609 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1080/2167] 10.222.0.45 () {58 vars in 1153 bytes} [Wed Aug 24 17:42:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:42:02.661660 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c31f7b-3042-4769-9e04-3b545c1dbb49 admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:42:02.664795 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1103ff-ead7-48c0-8813-ab4b43f42ab4 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:42:02.678680 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1103ff-ead7-48c0-8813-ab4b43f42ab4 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:42:02.680124 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c31f7b-3042-4769-9e04-3b545c1dbb49 admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:42:02.684310 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91268aab-782c-42f6-988b-74bd7922e64a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:42:02.685076 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91268aab-782c-42f6-988b-74bd7922e64a admin admin] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:02.685406 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63e7d7d-df28-45f9-99df-ae7d4d0154c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:42:02.685547 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91268aab-782c-42f6-988b-74bd7922e64a admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:02.686422 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91268aab-782c-42f6-988b-74bd7922e64a admin admin] PATH_INFO: `/v3/projects` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:02.686536 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63e7d7d-df28-45f9-99df-ae7d4d0154c9 admin admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:02.686536 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63e7d7d-df28-45f9-99df-ae7d4d0154c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:02.686798 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91268aab-782c-42f6-988b-74bd7922e64a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:42:02.686904 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63e7d7d-df28-45f9-99df-ae7d4d0154c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:02.687668 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63e7d7d-df28-45f9-99df-ae7d4d0154c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:42:02.689678 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63e7d7d-df28-45f9-99df-ae7d4d0154c9 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:42:02.690180 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91268aab-782c-42f6-988b-74bd7922e64a admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:42:02.701847 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1088/2168] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:42:02 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:42:02.705667 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63e7d7d-df28-45f9-99df-ae7d4d0154c9 admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:42:02.716850 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63e7d7d-df28-45f9-99df-ae7d4d0154c9 admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:42:02.721485 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80ff63e-2c33-4305-937d-db4d6d5c58d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:42:02.722143 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80ff63e-2c33-4305-937d-db4d6d5c58d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:02.722441 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80ff63e-2c33-4305-937d-db4d6d5c58d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:02.722700 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80ff63e-2c33-4305-937d-db4d6d5c58d2 admin admin] PATH_INFO: `/v3/projects/8f5f06c64c9a459fa45ba85855d4324f/users/757f4cb8bb714ccc90b2f2589f9b2a1e/roles/02476a5b82eb4b188c18a5d706a3ef68` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:02.724152 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1081/2169] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 17:42:02 2022] POST /identity/v3/projects => generated 375 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:42:02.725170 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91268aab-782c-42f6-988b-74bd7922e64a admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:42:02.738440 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80ff63e-2c33-4305-937d-db4d6d5c58d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f5f06c64c9a459fa45ba85855d4324f, user_id=757f4cb8bb714ccc90b2f2589f9b2a1e, role_id=02476a5b82eb4b188c18a5d706a3ef68)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:42:02.741110 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80ff63e-2c33-4305-937d-db4d6d5c58d2 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:42:02.753541 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1089/2170] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 17:42:02 2022] PUT /identity/v3/projects/8f5f06c64c9a459fa45ba85855d4324f/users/757f4cb8bb714ccc90b2f2589f9b2a1e/roles/02476a5b82eb4b188c18a5d706a3ef68 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:42:02.758432 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fc63f7-32ae-4b7b-892a-dc7caec62a38 None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:02.758719 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fc63f7-32ae-4b7b-892a-dc7caec62a38 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:02.758991 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fc63f7-32ae-4b7b-892a-dc7caec62a38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:02.771402 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=95,keystone:DELETE=3,keystone:INSERT=2 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:42:02.781970 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-49fc63f7-32ae-4b7b-892a-dc7caec62a38 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:42:02.783163 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-49fc63f7-32ae-4b7b-892a-dc7caec62a38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:02.791133 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91268aab-782c-42f6-988b-74bd7922e64a admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:42:02.848028 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1090/2171] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:42:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:42:02.854096 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637d7491-a905-4f53-b7db-760fb22e9d88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:42:02.854761 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637d7491-a905-4f53-b7db-760fb22e9d88 admin admin] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:02.855037 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637d7491-a905-4f53-b7db-760fb22e9d88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:02.855310 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637d7491-a905-4f53-b7db-760fb22e9d88 admin admin] PATH_INFO: `/v3/users` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:02.855985 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637d7491-a905-4f53-b7db-760fb22e9d88 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:42:02.857361 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fc63f7-32ae-4b7b-892a-dc7caec62a38 None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:42:02.857946 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637d7491-a905-4f53-b7db-760fb22e9d88 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:42:02.858810 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-49fc63f7-32ae-4b7b-892a-dc7caec62a38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:02.879281 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1082/2172] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 17:42:02 2022] POST /identity/v3/users => generated 288 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:42:02.883124 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637d7491-a905-4f53-b7db-760fb22e9d88 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:42:02.894093 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637d7491-a905-4f53-b7db-760fb22e9d88 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:42:02.898899 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac927df-ce66-43fb-a27e-4046e9e6bbc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:42:02.899507 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac927df-ce66-43fb-a27e-4046e9e6bbc4 admin admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:02.899765 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac927df-ce66-43fb-a27e-4046e9e6bbc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:02.900039 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac927df-ce66-43fb-a27e-4046e9e6bbc4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:02.900711 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac927df-ce66-43fb-a27e-4046e9e6bbc4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:42:02.903242 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac927df-ce66-43fb-a27e-4046e9e6bbc4 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:42:02.915848 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1083/2173] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:42:02 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 17:42:02.916921 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac927df-ce66-43fb-a27e-4046e9e6bbc4 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:42:02.917386 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fc63f7-32ae-4b7b-892a-dc7caec62a38 None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:42:02.921060 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25168455-cded-446b-bb03-47b209a0c60e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43ad98a250a144099ccd18378aedd4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:42:02.921671 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25168455-cded-446b-bb03-47b209a0c60e service neutron] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:02.921951 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25168455-cded-446b-bb03-47b209a0c60e service neutron] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:02.922223 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25168455-cded-446b-bb03-47b209a0c60e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:02.926876 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac927df-ce66-43fb-a27e-4046e9e6bbc4 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:42:02.932845 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870f7c22-c429-40b3-bea8-8b2d7ee1ce4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:42:02.933450 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870f7c22-c429-40b3-bea8-8b2d7ee1ce4d admin admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:02.933734 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870f7c22-c429-40b3-bea8-8b2d7ee1ce4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:02.934016 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870f7c22-c429-40b3-bea8-8b2d7ee1ce4d admin admin] PATH_INFO: `/v3/roles` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:02.934657 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870f7c22-c429-40b3-bea8-8b2d7ee1ce4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:42:02.936285 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870f7c22-c429-40b3-bea8-8b2d7ee1ce4d admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:42:02.937671 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25168455-cded-446b-bb03-47b209a0c60e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:42:02.939700 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25168455-cded-446b-bb03-47b209a0c60e service neutron] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:42:02.947465 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1084/2174] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:42:02 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:42:02.952513 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870f7c22-c429-40b3-bea8-8b2d7ee1ce4d admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:42:02.958385 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1091/2175] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:42:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 17:42:02.971092 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870f7c22-c429-40b3-bea8-8b2d7ee1ce4d admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:42:02.976378 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2c2dbd-dbec-40fb-9026-3a24f9583656 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:42:02.977067 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2c2dbd-dbec-40fb-9026-3a24f9583656 admin admin] REQUEST_METHOD: `PUT` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:02.977339 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2c2dbd-dbec-40fb-9026-3a24f9583656 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:02.977610 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2c2dbd-dbec-40fb-9026-3a24f9583656 admin admin] PATH_INFO: `/v3/projects/376329ca784b4fdea3205bf08edb45a8/users/ccb4c2a7b74a479d890b73a0aae33bad/roles/02476a5b82eb4b188c18a5d706a3ef68` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:02.995765 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2c2dbd-dbec-40fb-9026-3a24f9583656 admin admin] RBAC: Authorizing `identity:create_grant(project_id=376329ca784b4fdea3205bf08edb45a8, user_id=ccb4c2a7b74a479d890b73a0aae33bad, role_id=02476a5b82eb4b188c18a5d706a3ef68)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:42:02.998676 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2c2dbd-dbec-40fb-9026-3a24f9583656 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:42:03.012868 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1085/2176] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 17:42:02 2022] PUT /identity/v3/projects/376329ca784b4fdea3205bf08edb45a8/users/ccb4c2a7b74a479d890b73a0aae33bad/roles/02476a5b82eb4b188c18a5d706a3ef68 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:42:03.092738 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7fd5fd-77f6-4fba-be41-4db5ffed0085 None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:03.093110 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7fd5fd-77f6-4fba-be41-4db5ffed0085 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:03.093459 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7fd5fd-77f6-4fba-be41-4db5ffed0085 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:03.132631 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-5c7fd5fd-77f6-4fba-be41-4db5ffed0085 None None] MFA Rules not processed for user `ccb4c2a7b74a479d890b73a0aae33bad`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:42:03.133898 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-5c7fd5fd-77f6-4fba-be41-4db5ffed0085 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:03.139206 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:42:03.220182 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1092/2177] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:42:03 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:42:03.234247 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2c2dbd-dbec-40fb-9026-3a24f9583656 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:42:03.240733 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-6f2c2dbd-dbec-40fb-9026-3a24f9583656 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:03.311492 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2c2dbd-dbec-40fb-9026-3a24f9583656 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:42:03.315393 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5865e9-11a2-4c0b-a1f1-d01938aae01d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59be342541264942ad87f60ff4de6c8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:42:03.316000 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5865e9-11a2-4c0b-a1f1-d01938aae01d service cinder] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:03.316346 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5865e9-11a2-4c0b-a1f1-d01938aae01d service cinder] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:03.316690 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5865e9-11a2-4c0b-a1f1-d01938aae01d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:03.326054 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5865e9-11a2-4c0b-a1f1-d01938aae01d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:42:03.327760 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5865e9-11a2-4c0b-a1f1-d01938aae01d service cinder] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:42:03.339199 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1086/2178] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:42:03 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:42:03.973182 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfffa579-fe24-4b9a-bb8e-564ca3c9dba2 None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:03.973476 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfffa579-fe24-4b9a-bb8e-564ca3c9dba2 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:03.973763 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfffa579-fe24-4b9a-bb8e-564ca3c9dba2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:04.019284 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-cfffa579-fe24-4b9a-bb8e-564ca3c9dba2 None None] MFA Rules not processed for user `757f4cb8bb714ccc90b2f2589f9b2a1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:42:04.021015 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-cfffa579-fe24-4b9a-bb8e-564ca3c9dba2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:04.090651 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1093/2179] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:42:03 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:42:04.101416 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5865e9-11a2-4c0b-a1f1-d01938aae01d service cinder] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:42:04.110145 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5865e9-11a2-4c0b-a1f1-d01938aae01d service cinder] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:42:04.113824 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67297036-ce51-4517-bb6e-b7b36892617e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59be342541264942ad87f60ff4de6c8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:42:04.114424 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67297036-ce51-4517-bb6e-b7b36892617e service cinder] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:04.114700 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67297036-ce51-4517-bb6e-b7b36892617e service cinder] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:04.114972 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67297036-ce51-4517-bb6e-b7b36892617e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:04.127386 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67297036-ce51-4517-bb6e-b7b36892617e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:42:04.128903 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67297036-ce51-4517-bb6e-b7b36892617e service cinder] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:42:04.139598 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1087/2180] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:42:04 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:42:05.806601 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee055b9-718d-4baf-9069-ca3d92ea3152 None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:05.807028 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee055b9-718d-4baf-9069-ca3d92ea3152 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:05.807189 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee055b9-718d-4baf-9069-ca3d92ea3152 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:05.827900 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-8ee055b9-718d-4baf-9069-ca3d92ea3152 None None] MFA Rules not processed for user `ccb4c2a7b74a479d890b73a0aae33bad`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:42:05.829144 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-8ee055b9-718d-4baf-9069-ca3d92ea3152 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:05.835910 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1094/2181] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:42:05 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:42:06.554863 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73010838-5f14-4be1-aadc-53f05df4c8f5 None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:06.555298 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73010838-5f14-4be1-aadc-53f05df4c8f5 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:06.555472 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73010838-5f14-4be1-aadc-53f05df4c8f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:06.575241 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-73010838-5f14-4be1-aadc-53f05df4c8f5 None None] MFA Rules not processed for user `757f4cb8bb714ccc90b2f2589f9b2a1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:42:06.576447 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-73010838-5f14-4be1-aadc-53f05df4c8f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:06.583317 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1088/2182] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:42:06 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:42:10.798413 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12b3983-4d11-4e93-88d5-d49696c381a8 None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:10.799520 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12b3983-4d11-4e93-88d5-d49696c381a8 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:10.799995 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12b3983-4d11-4e93-88d5-d49696c381a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:10.826692 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-e12b3983-4d11-4e93-88d5-d49696c381a8 None None] MFA Rules not processed for user `af6710bfe62d4ba29dc48e2f7aba9ec5`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:42:10.828174 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-e12b3983-4d11-4e93-88d5-d49696c381a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:10.924883 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1095/2183] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:42:10 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:42:11.936672 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96da2fd-dc29-47b1-b24d-9f6936b965ee None None] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:11.937529 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96da2fd-dc29-47b1-b24d-9f6936b965ee None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:11.937961 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96da2fd-dc29-47b1-b24d-9f6936b965ee None None] PATH_INFO: `/` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:11.939607 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1089/2184] 10.222.0.45 () {54 vars in 921 bytes} [Wed Aug 24 17:42:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:42:11.969182 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad053ad5-b3c9-47a8-a9cb-687eb82e05fc None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:11.969755 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad053ad5-b3c9-47a8-a9cb-687eb82e05fc None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:11.969864 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad053ad5-b3c9-47a8-a9cb-687eb82e05fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:11.989662 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-ad053ad5-b3c9-47a8-a9cb-687eb82e05fc None None] MFA Rules not processed for user `37def4721b554b9ebb7704a19642d15b`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:42:11.990884 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-ad053ad5-b3c9-47a8-a9cb-687eb82e05fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:12.021654 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:42:12.063476 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1096/2185] 10.222.0.45 () {58 vars in 1018 bytes} [Wed Aug 24 17:42:11 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:42:12.093006 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96da2fd-dc29-47b1-b24d-9f6936b965ee None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:42:12.097772 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-c96da2fd-dc29-47b1-b24d-9f6936b965ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:12.119676 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad053ad5-b3c9-47a8-a9cb-687eb82e05fc None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:42:12.121408 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-ad053ad5-b3c9-47a8-a9cb-687eb82e05fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:12.134442 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96da2fd-dc29-47b1-b24d-9f6936b965ee None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:42:12.141637 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbbb7d5-7919-4ed5-b2bb-246adba0bf15 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37def4721b554b9ebb7704a19642d15b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:42:12.142255 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbbb7d5-7919-4ed5-b2bb-246adba0bf15 service placement] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:12.142531 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbbb7d5-7919-4ed5-b2bb-246adba0bf15 service placement] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:12.142804 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbbb7d5-7919-4ed5-b2bb-246adba0bf15 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:12.153792 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbbb7d5-7919-4ed5-b2bb-246adba0bf15 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:42:12.155829 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbbb7d5-7919-4ed5-b2bb-246adba0bf15 service placement] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:42:12.182121 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1090/2186] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:42:12 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:42:12.246573 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad053ad5-b3c9-47a8-a9cb-687eb82e05fc None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:42:12.252642 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5896017-9b36-4429-89be-7704d3d79cb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:42:12.253417 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5896017-9b36-4429-89be-7704d3d79cb5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:12.253670 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5896017-9b36-4429-89be-7704d3d79cb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:12.254281 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5896017-9b36-4429-89be-7704d3d79cb5 admin admin] PATH_INFO: `/v3/users/1d0ae308269b4cdd80990f7b38c0cd0c` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:12.255498 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5896017-9b36-4429-89be-7704d3d79cb5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1d0ae308269b4cdd80990f7b38c0cd0c)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:42:12.257505 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5896017-9b36-4429-89be-7704d3d79cb5 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:42:12.312942 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-a5896017-9b36-4429-89be-7704d3d79cb5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1d0ae308269b4cdd80990f7b38c0cd0c'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:42:12.313837 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-a5896017-9b36-4429-89be-7704d3d79cb5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1d0ae308269b4cdd80990f7b38c0cd0c'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:42:12.328265 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-a5896017-9b36-4429-89be-7704d3d79cb5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1d0ae308269b4cdd80990f7b38c0cd0c'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:42:12.330444 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-a5896017-9b36-4429-89be-7704d3d79cb5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1d0ae308269b4cdd80990f7b38c0cd0c'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:42:12.351192 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-a5896017-9b36-4429-89be-7704d3d79cb5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1d0ae308269b4cdd80990f7b38c0cd0c'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:42:12.371227 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1097/2187] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 17:42:12 2022] DELETE /identity/v3/users/1d0ae308269b4cdd80990f7b38c0cd0c => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:42:12.677096 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbbb7d5-7919-4ed5-b2bb-246adba0bf15 service placement] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:42:12.683120 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:42:12.690881 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbbb7d5-7919-4ed5-b2bb-246adba0bf15 service placement] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:42:12.772037 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0c79fa-4674-4bbc-adc2-16e120ebd470 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:42:12.772874 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0c79fa-4674-4bbc-adc2-16e120ebd470 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:12.773351 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0c79fa-4674-4bbc-adc2-16e120ebd470 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:12.773775 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0c79fa-4674-4bbc-adc2-16e120ebd470 admin admin] PATH_INFO: `/v3/projects/9206de8ce97b425cb5c675ec0a240d71` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:12.777410 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0c79fa-4674-4bbc-adc2-16e120ebd470 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9206de8ce97b425cb5c675ec0a240d71)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:42:12.779505 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0c79fa-4674-4bbc-adc2-16e120ebd470 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:42:12.843011 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-7d0c79fa-4674-4bbc-adc2-16e120ebd470 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9206de8ce97b425cb5c675ec0a240d71', 'request_id': 'req-7d0c79fa-4674-4bbc-adc2-16e120ebd470'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:42:12.850328 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-7d0c79fa-4674-4bbc-adc2-16e120ebd470 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9206de8ce97b425cb5c675ec0a240d71', 'request_id': 'req-7d0c79fa-4674-4bbc-adc2-16e120ebd470'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:42:12.861468 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-7d0c79fa-4674-4bbc-adc2-16e120ebd470 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9206de8ce97b425cb5c675ec0a240d71', 'request_id': 'req-7d0c79fa-4674-4bbc-adc2-16e120ebd470'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:42:12.885285 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-7d0c79fa-4674-4bbc-adc2-16e120ebd470 admin admin] The token cache is being invalidate because project 9206de8ce97b425cb5c675ec0a240d71 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81759) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:42:12.886148 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-7d0c79fa-4674-4bbc-adc2-16e120ebd470 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:42:12.888151 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-7d0c79fa-4674-4bbc-adc2-16e120ebd470 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:42:12.892587 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1091/2188] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 17:42:12 2022] DELETE /identity/v3/projects/9206de8ce97b425cb5c675ec0a240d71 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:42:12.895827 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b9af67-b702-4d9a-b2f9-8f60d62f402f None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:12.896180 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b9af67-b702-4d9a-b2f9-8f60d62f402f None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:12.896687 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b9af67-b702-4d9a-b2f9-8f60d62f402f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:12.926022 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-b5b9af67-b702-4d9a-b2f9-8f60d62f402f None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:42:12.927405 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-b5b9af67-b702-4d9a-b2f9-8f60d62f402f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:12.986056 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:42:13.015484 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1098/2189] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:42:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:42:13.023881 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8381b2-4e40-4bc3-92a0-35646a782e6d None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:13.024758 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8381b2-4e40-4bc3-92a0-35646a782e6d None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:13.025184 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8381b2-4e40-4bc3-92a0-35646a782e6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:13.046079 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-3c8381b2-4e40-4bc3-92a0-35646a782e6d None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:42:13.047554 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-3c8381b2-4e40-4bc3-92a0-35646a782e6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:13.056070 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1092/2190] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:42:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:42:13.061473 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b9af67-b702-4d9a-b2f9-8f60d62f402f None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:42:13.081452 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b9af67-b702-4d9a-b2f9-8f60d62f402f None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:42:13.086755 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954d576e-ffa9-4a21-897e-9cfa24c9fcb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:42:13.087543 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954d576e-ffa9-4a21-897e-9cfa24c9fcb1 admin admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:13.087810 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954d576e-ffa9-4a21-897e-9cfa24c9fcb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:13.088265 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954d576e-ffa9-4a21-897e-9cfa24c9fcb1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:13.089586 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954d576e-ffa9-4a21-897e-9cfa24c9fcb1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:42:13.089925 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954d576e-ffa9-4a21-897e-9cfa24c9fcb1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:42:13.091576 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954d576e-ffa9-4a21-897e-9cfa24c9fcb1 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:42:13.108215 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1099/2191] 10.222.0.45 () {58 vars in 1153 bytes} [Wed Aug 24 17:42:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:42:13.112895 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8381b2-4e40-4bc3-92a0-35646a782e6d None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:42:13.123349 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8381b2-4e40-4bc3-92a0-35646a782e6d None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:42:13.128213 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c13b99-1e88-4a0b-9df5-77ff905e0893 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:42:13.128850 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c13b99-1e88-4a0b-9df5-77ff905e0893 admin admin] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:13.129130 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c13b99-1e88-4a0b-9df5-77ff905e0893 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:13.129403 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c13b99-1e88-4a0b-9df5-77ff905e0893 admin admin] PATH_INFO: `/v3/projects` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:13.130215 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c13b99-1e88-4a0b-9df5-77ff905e0893 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:42:13.132138 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c13b99-1e88-4a0b-9df5-77ff905e0893 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:42:13.168157 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1093/2192] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 17:42:13 2022] POST /identity/v3/projects => generated 371 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:42:13.173612 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954d576e-ffa9-4a21-897e-9cfa24c9fcb1 admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:42:13.186338 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954d576e-ffa9-4a21-897e-9cfa24c9fcb1 admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:42:13.254884 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171a8539-7961-4ab2-9715-3e9365105733 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:42:13.255560 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171a8539-7961-4ab2-9715-3e9365105733 admin admin] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:13.255831 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171a8539-7961-4ab2-9715-3e9365105733 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:13.256111 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171a8539-7961-4ab2-9715-3e9365105733 admin admin] PATH_INFO: `/v3/users` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:13.256857 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171a8539-7961-4ab2-9715-3e9365105733 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:42:13.258990 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171a8539-7961-4ab2-9715-3e9365105733 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:42:13.285102 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1100/2193] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 17:42:13 2022] POST /identity/v3/users => generated 286 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:42:13.289176 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c13b99-1e88-4a0b-9df5-77ff905e0893 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:42:13.300126 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c13b99-1e88-4a0b-9df5-77ff905e0893 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:42:13.313982 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd29dd8e-e23e-4369-a47d-01e83dc8cf13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:42:13.315170 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd29dd8e-e23e-4369-a47d-01e83dc8cf13 admin admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:13.315518 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd29dd8e-e23e-4369-a47d-01e83dc8cf13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:13.315891 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd29dd8e-e23e-4369-a47d-01e83dc8cf13 admin admin] PATH_INFO: `/v3/roles` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:13.317464 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd29dd8e-e23e-4369-a47d-01e83dc8cf13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:42:13.319401 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd29dd8e-e23e-4369-a47d-01e83dc8cf13 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:42:13.339695 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1094/2194] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:42:13 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:42:13.343374 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171a8539-7961-4ab2-9715-3e9365105733 admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:42:13.356977 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171a8539-7961-4ab2-9715-3e9365105733 admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:42:13.376273 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0311979-3a9e-419b-ab87-bd1696b12877 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:42:13.378072 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0311979-3a9e-419b-ab87-bd1696b12877 admin admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:13.379538 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0311979-3a9e-419b-ab87-bd1696b12877 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:13.380039 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0311979-3a9e-419b-ab87-bd1696b12877 admin admin] PATH_INFO: `/v3/roles` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:13.381506 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0311979-3a9e-419b-ab87-bd1696b12877 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:42:13.383980 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0311979-3a9e-419b-ab87-bd1696b12877 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:42:13.409110 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1101/2195] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:42:13 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:42:13.409785 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd29dd8e-e23e-4369-a47d-01e83dc8cf13 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:42:13.427055 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd29dd8e-e23e-4369-a47d-01e83dc8cf13 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:42:13.433049 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d8f2ab-869b-4623-a0db-70b7a5e58441 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:42:13.433934 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d8f2ab-869b-4623-a0db-70b7a5e58441 admin admin] REQUEST_METHOD: `PUT` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:13.434296 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d8f2ab-869b-4623-a0db-70b7a5e58441 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:13.434666 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d8f2ab-869b-4623-a0db-70b7a5e58441 admin admin] PATH_INFO: `/v3/projects/3ae0a93a58284227973f5eab0a220036/users/91353928ffce4278a8a72cf25f3296f2/roles/02476a5b82eb4b188c18a5d706a3ef68` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:13.453271 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d8f2ab-869b-4623-a0db-70b7a5e58441 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ae0a93a58284227973f5eab0a220036, user_id=91353928ffce4278a8a72cf25f3296f2, role_id=02476a5b82eb4b188c18a5d706a3ef68)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:42:13.457170 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d8f2ab-869b-4623-a0db-70b7a5e58441 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:42:13.487236 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1095/2196] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 17:42:13 2022] PUT /identity/v3/projects/3ae0a93a58284227973f5eab0a220036/users/91353928ffce4278a8a72cf25f3296f2/roles/02476a5b82eb4b188c18a5d706a3ef68 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:42:13.507798 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84b0c7a-f522-458a-ade7-20e351203e25 None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:13.508153 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84b0c7a-f522-458a-ade7-20e351203e25 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:13.508487 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84b0c7a-f522-458a-ade7-20e351203e25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:13.562688 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-e84b0c7a-f522-458a-ade7-20e351203e25 None None] MFA Rules not processed for user `91353928ffce4278a8a72cf25f3296f2`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:42:13.564538 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-e84b0c7a-f522-458a-ade7-20e351203e25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:13.649520 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1102/2197] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:42:13 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:42:13.654337 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d8f2ab-869b-4623-a0db-70b7a5e58441 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:42:13.690334 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:42:13.724063 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d8f2ab-869b-4623-a0db-70b7a5e58441 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:42:13.728836 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3f742f-1464-469a-8d59-0ecb3ec192de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:42:13.730031 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3f742f-1464-469a-8d59-0ecb3ec192de admin admin] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:13.730308 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3f742f-1464-469a-8d59-0ecb3ec192de admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:13.730580 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3f742f-1464-469a-8d59-0ecb3ec192de admin admin] PATH_INFO: `/v3/projects` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:13.731214 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3f742f-1464-469a-8d59-0ecb3ec192de admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:42:13.733147 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3f742f-1464-469a-8d59-0ecb3ec192de admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:42:13.771738 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1096/2198] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 17:42:13 2022] POST /identity/v3/projects => generated 373 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:42:13.777307 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84b0c7a-f522-458a-ade7-20e351203e25 None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:42:13.789398 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84b0c7a-f522-458a-ade7-20e351203e25 None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:42:13.838505 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:42:13.854600 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12e2212-bb2d-4a6c-b047-e6c6eb3ef9f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:42:13.855218 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12e2212-bb2d-4a6c-b047-e6c6eb3ef9f4 admin admin] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:13.855500 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12e2212-bb2d-4a6c-b047-e6c6eb3ef9f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:13.855775 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12e2212-bb2d-4a6c-b047-e6c6eb3ef9f4 admin admin] PATH_INFO: `/v3/users` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:13.856504 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12e2212-bb2d-4a6c-b047-e6c6eb3ef9f4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:42:13.858502 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12e2212-bb2d-4a6c-b047-e6c6eb3ef9f4 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:42:13.880585 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1103/2199] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 17:42:13 2022] POST /identity/v3/users => generated 293 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:42:13.884022 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3f742f-1464-469a-8d59-0ecb3ec192de admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:42:13.895244 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3f742f-1464-469a-8d59-0ecb3ec192de admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:42:13.900045 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1198f5-2325-4cf5-aa62-0e80af75ab1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:42:13.900773 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1198f5-2325-4cf5-aa62-0e80af75ab1f admin admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:13.901799 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1198f5-2325-4cf5-aa62-0e80af75ab1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:13.902143 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1198f5-2325-4cf5-aa62-0e80af75ab1f admin admin] PATH_INFO: `/v3/roles` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:13.902865 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1198f5-2325-4cf5-aa62-0e80af75ab1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:42:13.904544 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1198f5-2325-4cf5-aa62-0e80af75ab1f admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:42:13.915274 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1097/2200] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:42:13 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:42:13.920095 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12e2212-bb2d-4a6c-b047-e6c6eb3ef9f4 admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:42:13.931230 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12e2212-bb2d-4a6c-b047-e6c6eb3ef9f4 admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:42:13.936913 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dedbb8-c14e-4454-b205-80ffe5601a5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:42:13.937781 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dedbb8-c14e-4454-b205-80ffe5601a5c admin admin] REQUEST_METHOD: `PUT` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:13.938062 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dedbb8-c14e-4454-b205-80ffe5601a5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:13.938334 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dedbb8-c14e-4454-b205-80ffe5601a5c admin admin] PATH_INFO: `/v3/projects/6184892a7e03485e8cc723f01c9ff067/users/22159e1b676942818c2774472fe536ba/roles/e411ad9eb96e4e8d85b59b5b700ada7b` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:13.953940 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24dedbb8-c14e-4454-b205-80ffe5601a5c admin admin] RBAC: Authorizing `identity:create_grant(project_id=6184892a7e03485e8cc723f01c9ff067, user_id=22159e1b676942818c2774472fe536ba, role_id=e411ad9eb96e4e8d85b59b5b700ada7b)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:42:13.957080 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24dedbb8-c14e-4454-b205-80ffe5601a5c admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:42:13.970629 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1104/2201] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 17:42:13 2022] PUT /identity/v3/projects/6184892a7e03485e8cc723f01c9ff067/users/22159e1b676942818c2774472fe536ba/roles/e411ad9eb96e4e8d85b59b5b700ada7b => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:42:13.974812 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1198f5-2325-4cf5-aa62-0e80af75ab1f admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:42:13.986312 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1198f5-2325-4cf5-aa62-0e80af75ab1f admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:42:14.051213 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4102f2-15b5-4904-93d2-82bc0dfc6449 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:42:14.051827 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4102f2-15b5-4904-93d2-82bc0dfc6449 admin admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:14.052099 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4102f2-15b5-4904-93d2-82bc0dfc6449 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:14.052532 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4102f2-15b5-4904-93d2-82bc0dfc6449 admin admin] PATH_INFO: `/v3/roles` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:14.053222 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4102f2-15b5-4904-93d2-82bc0dfc6449 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:42:14.054927 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4102f2-15b5-4904-93d2-82bc0dfc6449 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:42:14.076413 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1098/2202] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:42:13 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:42:14.076510 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dedbb8-c14e-4454-b205-80ffe5601a5c admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:42:14.082171 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dedbb8-c14e-4454-b205-80ffe5601a5c admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:42:14.087155 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4167ff9e-2cc5-40c5-88d4-972f093c1c1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:42:14.087952 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4167ff9e-2cc5-40c5-88d4-972f093c1c1b admin admin] REQUEST_METHOD: `PUT` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:14.088237 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4167ff9e-2cc5-40c5-88d4-972f093c1c1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:14.088571 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4167ff9e-2cc5-40c5-88d4-972f093c1c1b admin admin] PATH_INFO: `/v3/projects/6184892a7e03485e8cc723f01c9ff067/users/22159e1b676942818c2774472fe536ba/roles/e411ad9eb96e4e8d85b59b5b700ada7b` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:14.090578 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4167ff9e-2cc5-40c5-88d4-972f093c1c1b admin admin] RBAC: Authorizing `identity:create_grant(project_id=6184892a7e03485e8cc723f01c9ff067, user_id=22159e1b676942818c2774472fe536ba, role_id=e411ad9eb96e4e8d85b59b5b700ada7b)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:42:14.093252 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4167ff9e-2cc5-40c5-88d4-972f093c1c1b admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:42:14.105012 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1105/2203] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 17:42:14 2022] PUT /identity/v3/projects/6184892a7e03485e8cc723f01c9ff067/users/22159e1b676942818c2774472fe536ba/roles/e411ad9eb96e4e8d85b59b5b700ada7b => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:42:14.117410 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629bb5c6-6d10-48d3-ba8f-23785c957437 None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:14.117719 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629bb5c6-6d10-48d3-ba8f-23785c957437 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:14.118259 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629bb5c6-6d10-48d3-ba8f-23785c957437 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:14.146575 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-629bb5c6-6d10-48d3-ba8f-23785c957437 None None] MFA Rules not processed for user `22159e1b676942818c2774472fe536ba`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:42:14.147918 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-629bb5c6-6d10-48d3-ba8f-23785c957437 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:14.214936 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1099/2204] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:42:14 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:42:14.226913 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4167ff9e-2cc5-40c5-88d4-972f093c1c1b admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:42:14.228622 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-4167ff9e-2cc5-40c5-88d4-972f093c1c1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:14.288243 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4167ff9e-2cc5-40c5-88d4-972f093c1c1b admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:42:14.292499 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9bbcc5-0788-4893-9387-5256243d51aa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59be342541264942ad87f60ff4de6c8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:42:14.293117 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9bbcc5-0788-4893-9387-5256243d51aa service cinder] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:14.293399 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9bbcc5-0788-4893-9387-5256243d51aa service cinder] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:14.293674 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9bbcc5-0788-4893-9387-5256243d51aa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:14.302565 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9bbcc5-0788-4893-9387-5256243d51aa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:42:14.304064 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9bbcc5-0788-4893-9387-5256243d51aa service cinder] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:42:14.313358 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1106/2205] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:42:14 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:42:14.713209 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629bb5c6-6d10-48d3-ba8f-23785c957437 None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:42:14.740598 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:42:14.780922 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629bb5c6-6d10-48d3-ba8f-23785c957437 None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:42:14.785520 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c339c2-9ebc-44dd-87f5-f440c3db8ba2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:42:14.786150 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c339c2-9ebc-44dd-87f5-f440c3db8ba2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:14.786428 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c339c2-9ebc-44dd-87f5-f440c3db8ba2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:14.786704 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c339c2-9ebc-44dd-87f5-f440c3db8ba2 admin admin] PATH_INFO: `/v3/users/91353928ffce4278a8a72cf25f3296f2` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:14.787825 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c339c2-9ebc-44dd-87f5-f440c3db8ba2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=91353928ffce4278a8a72cf25f3296f2)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:42:14.789772 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c339c2-9ebc-44dd-87f5-f440c3db8ba2 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:42:14.837221 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-88c339c2-9ebc-44dd-87f5-f440c3db8ba2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '91353928ffce4278a8a72cf25f3296f2'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:42:14.837888 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-88c339c2-9ebc-44dd-87f5-f440c3db8ba2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '91353928ffce4278a8a72cf25f3296f2'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:42:14.845637 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-88c339c2-9ebc-44dd-87f5-f440c3db8ba2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '91353928ffce4278a8a72cf25f3296f2'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:42:14.846276 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-88c339c2-9ebc-44dd-87f5-f440c3db8ba2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '91353928ffce4278a8a72cf25f3296f2'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:42:14.853582 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-88c339c2-9ebc-44dd-87f5-f440c3db8ba2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '91353928ffce4278a8a72cf25f3296f2'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:42:14.871405 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1100/2206] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 17:42:14 2022] DELETE /identity/v3/users/91353928ffce4278a8a72cf25f3296f2 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:42:14.875852 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1e44a7-0b4d-446c-92b4-7221947ddca3 None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:14.876139 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1e44a7-0b4d-446c-92b4-7221947ddca3 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:14.876416 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1e44a7-0b4d-446c-92b4-7221947ddca3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:14.896224 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-0e1e44a7-0b4d-446c-92b4-7221947ddca3 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:42:14.897674 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-0e1e44a7-0b4d-446c-92b4-7221947ddca3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:14.959015 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1107/2207] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:42:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:42:14.968525 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c339c2-9ebc-44dd-87f5-f440c3db8ba2 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:42:14.969954 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-88c339c2-9ebc-44dd-87f5-f440c3db8ba2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:15.025996 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c339c2-9ebc-44dd-87f5-f440c3db8ba2 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:42:15.029602 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8c1462-759d-40ca-b651-f64041b57df6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43ad98a250a144099ccd18378aedd4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:42:15.030199 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8c1462-759d-40ca-b651-f64041b57df6 service neutron] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:15.030482 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8c1462-759d-40ca-b651-f64041b57df6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:15.030758 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8c1462-759d-40ca-b651-f64041b57df6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:15.039554 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8c1462-759d-40ca-b651-f64041b57df6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:42:15.041024 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8c1462-759d-40ca-b651-f64041b57df6 service neutron] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:42:15.050282 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1101/2208] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:42:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:42:15.318767 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1e44a7-0b4d-446c-92b4-7221947ddca3 None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:42:15.320349 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-0e1e44a7-0b4d-446c-92b4-7221947ddca3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:15.337928 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1e44a7-0b4d-446c-92b4-7221947ddca3 None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:42:15.343977 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f5fc6a-a4af-4db4-9bda-0cefca7c00fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:42:15.344659 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f5fc6a-a4af-4db4-9bda-0cefca7c00fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:15.344949 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f5fc6a-a4af-4db4-9bda-0cefca7c00fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:15.345230 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f5fc6a-a4af-4db4-9bda-0cefca7c00fd admin admin] PATH_INFO: `/v3/projects/3ae0a93a58284227973f5eab0a220036` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:15.346433 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f5fc6a-a4af-4db4-9bda-0cefca7c00fd admin admin] RBAC: Authorizing `identity:delete_project(project_id=3ae0a93a58284227973f5eab0a220036)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:42:15.348350 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f5fc6a-a4af-4db4-9bda-0cefca7c00fd admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:42:15.385261 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=93,keystone:INSERT=5,keystone:DELETE=2 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:42:15.397123 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-b6f5fc6a-a4af-4db4-9bda-0cefca7c00fd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3ae0a93a58284227973f5eab0a220036', 'request_id': 'req-b6f5fc6a-a4af-4db4-9bda-0cefca7c00fd'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:42:15.404811 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-b6f5fc6a-a4af-4db4-9bda-0cefca7c00fd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3ae0a93a58284227973f5eab0a220036', 'request_id': 'req-b6f5fc6a-a4af-4db4-9bda-0cefca7c00fd'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:42:15.411193 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-b6f5fc6a-a4af-4db4-9bda-0cefca7c00fd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3ae0a93a58284227973f5eab0a220036', 'request_id': 'req-b6f5fc6a-a4af-4db4-9bda-0cefca7c00fd'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:42:15.421871 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-b6f5fc6a-a4af-4db4-9bda-0cefca7c00fd admin admin] The token cache is being invalidate because project 3ae0a93a58284227973f5eab0a220036 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81758) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:42:15.422165 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-b6f5fc6a-a4af-4db4-9bda-0cefca7c00fd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:42:15.422799 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-b6f5fc6a-a4af-4db4-9bda-0cefca7c00fd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:42:15.424265 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1108/2209] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 17:42:15 2022] DELETE /identity/v3/projects/3ae0a93a58284227973f5eab0a220036 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:42:15.428272 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8c1462-759d-40ca-b651-f64041b57df6 service neutron] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:42:15.438488 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8c1462-759d-40ca-b651-f64041b57df6 service neutron] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:42:15.482486 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:42:15.497361 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df05b97c-b841-4f16-9ef4-684e97449b29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:42:15.497993 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df05b97c-b841-4f16-9ef4-684e97449b29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:15.498275 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df05b97c-b841-4f16-9ef4-684e97449b29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:15.498550 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df05b97c-b841-4f16-9ef4-684e97449b29 admin admin] PATH_INFO: `/v3/users/22159e1b676942818c2774472fe536ba` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:15.499670 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df05b97c-b841-4f16-9ef4-684e97449b29 admin admin] RBAC: Authorizing `identity:delete_user(user_id=22159e1b676942818c2774472fe536ba)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:42:15.501636 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df05b97c-b841-4f16-9ef4-684e97449b29 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:42:15.547631 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-df05b97c-b841-4f16-9ef4-684e97449b29 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '22159e1b676942818c2774472fe536ba'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:42:15.548262 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-df05b97c-b841-4f16-9ef4-684e97449b29 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '22159e1b676942818c2774472fe536ba'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:42:15.555195 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-df05b97c-b841-4f16-9ef4-684e97449b29 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '22159e1b676942818c2774472fe536ba'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:42:15.555819 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-df05b97c-b841-4f16-9ef4-684e97449b29 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '22159e1b676942818c2774472fe536ba'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:42:15.563411 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-df05b97c-b841-4f16-9ef4-684e97449b29 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '22159e1b676942818c2774472fe536ba'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:42:15.580022 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1102/2210] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 17:42:15 2022] DELETE /identity/v3/users/22159e1b676942818c2774472fe536ba => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:42:15.984300 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f5fc6a-a4af-4db4-9bda-0cefca7c00fd admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:42:15.985973 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-b6f5fc6a-a4af-4db4-9bda-0cefca7c00fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:15.999290 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f5fc6a-a4af-4db4-9bda-0cefca7c00fd admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:42:16.061280 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8346b7ee-0cbb-44fd-8ecb-2d804e2aeef0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:42:16.062012 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8346b7ee-0cbb-44fd-8ecb-2d804e2aeef0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:16.062363 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8346b7ee-0cbb-44fd-8ecb-2d804e2aeef0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:16.062709 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8346b7ee-0cbb-44fd-8ecb-2d804e2aeef0 admin admin] PATH_INFO: `/v3/projects/6184892a7e03485e8cc723f01c9ff067` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:16.064139 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8346b7ee-0cbb-44fd-8ecb-2d804e2aeef0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6184892a7e03485e8cc723f01c9ff067)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:42:16.066158 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8346b7ee-0cbb-44fd-8ecb-2d804e2aeef0 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:42:16.113730 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-8346b7ee-0cbb-44fd-8ecb-2d804e2aeef0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6184892a7e03485e8cc723f01c9ff067', 'request_id': 'req-8346b7ee-0cbb-44fd-8ecb-2d804e2aeef0'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:42:16.120469 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-8346b7ee-0cbb-44fd-8ecb-2d804e2aeef0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6184892a7e03485e8cc723f01c9ff067', 'request_id': 'req-8346b7ee-0cbb-44fd-8ecb-2d804e2aeef0'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:42:16.126593 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-8346b7ee-0cbb-44fd-8ecb-2d804e2aeef0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6184892a7e03485e8cc723f01c9ff067', 'request_id': 'req-8346b7ee-0cbb-44fd-8ecb-2d804e2aeef0'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:42:16.137493 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-8346b7ee-0cbb-44fd-8ecb-2d804e2aeef0 admin admin] The token cache is being invalidate because project 6184892a7e03485e8cc723f01c9ff067 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81758) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:42:16.137785 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-8346b7ee-0cbb-44fd-8ecb-2d804e2aeef0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:42:16.138418 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-8346b7ee-0cbb-44fd-8ecb-2d804e2aeef0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:42:16.139884 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1109/2211] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 17:42:15 2022] DELETE /identity/v3/projects/6184892a7e03485e8cc723f01c9ff067 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:42:16.146166 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85d0fe2-e2bf-4042-a29d-4c34edfbe479 None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:16.146531 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85d0fe2-e2bf-4042-a29d-4c34edfbe479 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:16.146879 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85d0fe2-e2bf-4042-a29d-4c34edfbe479 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:16.167504 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-a85d0fe2-e2bf-4042-a29d-4c34edfbe479 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:42:16.168739 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-a85d0fe2-e2bf-4042-a29d-4c34edfbe479 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:16.229402 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1103/2212] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:42:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:42:16.236947 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e292c1da-b547-4cca-acec-0c86ad8e9cec None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:16.237231 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e292c1da-b547-4cca-acec-0c86ad8e9cec None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:16.237503 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e292c1da-b547-4cca-acec-0c86ad8e9cec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:16.261626 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-e292c1da-b547-4cca-acec-0c86ad8e9cec None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:42:16.262826 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-e292c1da-b547-4cca-acec-0c86ad8e9cec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:16.270908 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1110/2213] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:42:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:42:16.275332 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85d0fe2-e2bf-4042-a29d-4c34edfbe479 None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:42:16.285256 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85d0fe2-e2bf-4042-a29d-4c34edfbe479 None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:42:16.291583 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24641d96-fb89-45c6-bd8a-09f2c0430525 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:42:16.292450 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24641d96-fb89-45c6-bd8a-09f2c0430525 admin admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:16.292807 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24641d96-fb89-45c6-bd8a-09f2c0430525 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:16.293277 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24641d96-fb89-45c6-bd8a-09f2c0430525 admin admin] PATH_INFO: `/v3/domains` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:16.293998 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24641d96-fb89-45c6-bd8a-09f2c0430525 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:42:16.294300 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24641d96-fb89-45c6-bd8a-09f2c0430525 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:42:16.295889 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24641d96-fb89-45c6-bd8a-09f2c0430525 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:42:16.311012 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1104/2214] 10.222.0.45 () {58 vars in 1153 bytes} [Wed Aug 24 17:42:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:42:16.312447 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e292c1da-b547-4cca-acec-0c86ad8e9cec None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:42:16.322536 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e292c1da-b547-4cca-acec-0c86ad8e9cec None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:42:16.328319 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14574316-13ba-4417-a0ce-c1cb47089b6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:42:16.328820 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14574316-13ba-4417-a0ce-c1cb47089b6f admin admin] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:16.329103 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14574316-13ba-4417-a0ce-c1cb47089b6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:16.329385 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14574316-13ba-4417-a0ce-c1cb47089b6f admin admin] PATH_INFO: `/v3/projects` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:16.330017 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14574316-13ba-4417-a0ce-c1cb47089b6f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:42:16.331929 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14574316-13ba-4417-a0ce-c1cb47089b6f admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:42:16.363785 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1111/2215] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 17:42:16 2022] POST /identity/v3/projects => generated 373 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:42:16.367928 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24641d96-fb89-45c6-bd8a-09f2c0430525 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:42:16.377766 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24641d96-fb89-45c6-bd8a-09f2c0430525 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:42:16.426908 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:42:16.437758 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92b5ad9-c099-4fde-9478-36a64420d620 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:42:16.438361 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92b5ad9-c099-4fde-9478-36a64420d620 admin admin] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:16.438645 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92b5ad9-c099-4fde-9478-36a64420d620 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:16.438948 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92b5ad9-c099-4fde-9478-36a64420d620 admin admin] PATH_INFO: `/v3/users` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:16.439589 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92b5ad9-c099-4fde-9478-36a64420d620 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:42:16.441560 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92b5ad9-c099-4fde-9478-36a64420d620 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:42:16.463734 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1105/2216] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 17:42:16 2022] POST /identity/v3/users => generated 287 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:42:16.467894 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14574316-13ba-4417-a0ce-c1cb47089b6f admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:42:16.478252 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14574316-13ba-4417-a0ce-c1cb47089b6f admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:42:16.482998 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f13a93-ca17-4a5f-a26a-c338f248453c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:42:16.483660 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f13a93-ca17-4a5f-a26a-c338f248453c admin admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:16.483946 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f13a93-ca17-4a5f-a26a-c338f248453c admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:16.484226 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f13a93-ca17-4a5f-a26a-c338f248453c admin admin] PATH_INFO: `/v3/roles` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:16.485036 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f13a93-ca17-4a5f-a26a-c338f248453c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:42:16.486640 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f13a93-ca17-4a5f-a26a-c338f248453c admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:42:16.497679 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1112/2217] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:42:16 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 17:42:16.501833 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92b5ad9-c099-4fde-9478-36a64420d620 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:42:16.512512 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92b5ad9-c099-4fde-9478-36a64420d620 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:42:16.517536 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5153ae8-210f-48e0-a85a-5371e6781838 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:42:16.518134 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5153ae8-210f-48e0-a85a-5371e6781838 admin admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:16.518444 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5153ae8-210f-48e0-a85a-5371e6781838 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:16.518722 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5153ae8-210f-48e0-a85a-5371e6781838 admin admin] PATH_INFO: `/v3/roles` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:16.519365 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5153ae8-210f-48e0-a85a-5371e6781838 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:42:16.521077 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5153ae8-210f-48e0-a85a-5371e6781838 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:42:16.531584 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1106/2218] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:42:16 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:42:16.535737 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f13a93-ca17-4a5f-a26a-c338f248453c admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:42:16.545622 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f13a93-ca17-4a5f-a26a-c338f248453c admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:42:16.550205 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d1dfc0-630b-404e-bbee-12dfebdaeaab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:42:16.550849 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d1dfc0-630b-404e-bbee-12dfebdaeaab admin admin] REQUEST_METHOD: `PUT` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:16.551128 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d1dfc0-630b-404e-bbee-12dfebdaeaab admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:16.551402 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d1dfc0-630b-404e-bbee-12dfebdaeaab admin admin] PATH_INFO: `/v3/projects/04d2d2cf5a2849a88cbfa9885ee2d492/users/5d9926174a044ba4a36960c2c64ec5a9/roles/02476a5b82eb4b188c18a5d706a3ef68` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:16.562726 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:DELETE=12,keystone:SELECT=85,keystone:INSERT=3 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:42:16.567104 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d1dfc0-630b-404e-bbee-12dfebdaeaab admin admin] RBAC: Authorizing `identity:create_grant(project_id=04d2d2cf5a2849a88cbfa9885ee2d492, user_id=5d9926174a044ba4a36960c2c64ec5a9, role_id=02476a5b82eb4b188c18a5d706a3ef68)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:42:16.569911 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d1dfc0-630b-404e-bbee-12dfebdaeaab admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:42:16.584341 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1113/2219] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 17:42:16 2022] PUT /identity/v3/projects/04d2d2cf5a2849a88cbfa9885ee2d492/users/5d9926174a044ba4a36960c2c64ec5a9/roles/02476a5b82eb4b188c18a5d706a3ef68 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:42:16.589231 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cbdf59-c291-497b-bab2-91d5f25ad8ae None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:16.589516 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cbdf59-c291-497b-bab2-91d5f25ad8ae None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:16.589789 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cbdf59-c291-497b-bab2-91d5f25ad8ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:16.607975 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-26cbdf59-c291-497b-bab2-91d5f25ad8ae None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:42:16.609135 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-26cbdf59-c291-497b-bab2-91d5f25ad8ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:16.666515 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1107/2220] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:42:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:42:16.675993 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d1dfc0-630b-404e-bbee-12dfebdaeaab admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:42:16.677743 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-e6d1dfc0-630b-404e-bbee-12dfebdaeaab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:16.733989 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d1dfc0-630b-404e-bbee-12dfebdaeaab admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:42:16.737736 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e87950e-43b9-4c65-8d76-470d6b7eda18 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43ad98a250a144099ccd18378aedd4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:42:16.738373 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e87950e-43b9-4c65-8d76-470d6b7eda18 service neutron] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:16.738657 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e87950e-43b9-4c65-8d76-470d6b7eda18 service neutron] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:16.738958 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e87950e-43b9-4c65-8d76-470d6b7eda18 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:16.748988 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e87950e-43b9-4c65-8d76-470d6b7eda18 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:42:16.750426 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e87950e-43b9-4c65-8d76-470d6b7eda18 service neutron] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:42:16.759857 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1114/2221] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:42:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 17:42:19.181521 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec3f43a-a5d2-420d-acf7-9472c165b653 None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:19.182052 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec3f43a-a5d2-420d-acf7-9472c165b653 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:19.183056 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec3f43a-a5d2-420d-acf7-9472c165b653 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:19.215166 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-7ec3f43a-a5d2-420d-acf7-9472c165b653 None None] MFA Rules not processed for user `5d9926174a044ba4a36960c2c64ec5a9`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:42:19.216832 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-7ec3f43a-a5d2-420d-acf7-9472c165b653 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:19.346722 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1108/2222] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:42:19 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:42:19.352051 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e87950e-43b9-4c65-8d76-470d6b7eda18 service neutron] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:42:19.366260 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e87950e-43b9-4c65-8d76-470d6b7eda18 service neutron] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:42:19.383481 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea35805-809b-415b-a654-f083dc3ba01c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:42:19.384100 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea35805-809b-415b-a654-f083dc3ba01c admin admin] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:19.384378 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea35805-809b-415b-a654-f083dc3ba01c admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:19.384680 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea35805-809b-415b-a654-f083dc3ba01c admin admin] PATH_INFO: `/v3/projects` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:19.385369 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea35805-809b-415b-a654-f083dc3ba01c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:42:19.387282 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea35805-809b-415b-a654-f083dc3ba01c admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:42:19.434126 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1115/2223] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 17:42:19 2022] POST /identity/v3/projects => generated 373 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:42:19.438612 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec3f43a-a5d2-420d-acf7-9472c165b653 None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:42:19.451962 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec3f43a-a5d2-420d-acf7-9472c165b653 None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:42:19.515866 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9eb4fa-e759-4b9c-9f39-a2706015ff37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:42:19.516691 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9eb4fa-e759-4b9c-9f39-a2706015ff37 admin admin] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:19.517078 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9eb4fa-e759-4b9c-9f39-a2706015ff37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:19.517448 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9eb4fa-e759-4b9c-9f39-a2706015ff37 admin admin] PATH_INFO: `/v3/users` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:19.518174 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9eb4fa-e759-4b9c-9f39-a2706015ff37 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:42:19.520253 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9eb4fa-e759-4b9c-9f39-a2706015ff37 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:42:19.530059 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:42:19.549935 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1109/2224] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 17:42:19 2022] POST /identity/v3/users => generated 293 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:42:19.551424 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea35805-809b-415b-a654-f083dc3ba01c admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:42:19.563800 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea35805-809b-415b-a654-f083dc3ba01c admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:42:19.571489 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43d6fa6-ecef-4515-8833-de8cd876732c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:42:19.572448 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43d6fa6-ecef-4515-8833-de8cd876732c admin admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:19.573020 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43d6fa6-ecef-4515-8833-de8cd876732c admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:19.573346 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43d6fa6-ecef-4515-8833-de8cd876732c admin admin] PATH_INFO: `/v3/roles` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:19.587229 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43d6fa6-ecef-4515-8833-de8cd876732c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:42:19.587229 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43d6fa6-ecef-4515-8833-de8cd876732c admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:42:19.590214 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1116/2225] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:42:19 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:42:19.594588 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9eb4fa-e759-4b9c-9f39-a2706015ff37 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:42:19.606672 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9eb4fa-e759-4b9c-9f39-a2706015ff37 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:42:19.611468 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86f35ef-c080-40ea-a865-33612701588f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:42:19.612122 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86f35ef-c080-40ea-a865-33612701588f admin admin] REQUEST_METHOD: `PUT` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:19.612400 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86f35ef-c080-40ea-a865-33612701588f admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:19.612770 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86f35ef-c080-40ea-a865-33612701588f admin admin] PATH_INFO: `/v3/projects/98cbea710e7043648dbe98cd04885757/users/d512d56d7e434a3dbc11c77ce6c69961/roles/e411ad9eb96e4e8d85b59b5b700ada7b` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:19.630581 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86f35ef-c080-40ea-a865-33612701588f admin admin] RBAC: Authorizing `identity:create_grant(project_id=98cbea710e7043648dbe98cd04885757, user_id=d512d56d7e434a3dbc11c77ce6c69961, role_id=e411ad9eb96e4e8d85b59b5b700ada7b)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:42:19.634040 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86f35ef-c080-40ea-a865-33612701588f admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:42:19.651507 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1110/2226] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 17:42:19 2022] PUT /identity/v3/projects/98cbea710e7043648dbe98cd04885757/users/d512d56d7e434a3dbc11c77ce6c69961/roles/e411ad9eb96e4e8d85b59b5b700ada7b => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:42:19.655074 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43d6fa6-ecef-4515-8833-de8cd876732c admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:42:19.668177 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43d6fa6-ecef-4515-8833-de8cd876732c admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:42:19.728175 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4178a10-3c9f-4ac8-b1f4-15ca304d4941 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:42:19.728839 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4178a10-3c9f-4ac8-b1f4-15ca304d4941 admin admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:19.729718 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4178a10-3c9f-4ac8-b1f4-15ca304d4941 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:19.730131 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4178a10-3c9f-4ac8-b1f4-15ca304d4941 admin admin] PATH_INFO: `/v3/roles` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:19.730813 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4178a10-3c9f-4ac8-b1f4-15ca304d4941 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:42:19.732565 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4178a10-3c9f-4ac8-b1f4-15ca304d4941 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:42:19.744474 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1117/2227] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:42:19 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:42:19.748733 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86f35ef-c080-40ea-a865-33612701588f admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:42:19.759363 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86f35ef-c080-40ea-a865-33612701588f admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:42:19.764582 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116e3013-5ae3-4fbf-bef8-bcb7299d5506 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:42:19.765836 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116e3013-5ae3-4fbf-bef8-bcb7299d5506 admin admin] REQUEST_METHOD: `PUT` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:19.766120 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116e3013-5ae3-4fbf-bef8-bcb7299d5506 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:19.766393 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116e3013-5ae3-4fbf-bef8-bcb7299d5506 admin admin] PATH_INFO: `/v3/projects/98cbea710e7043648dbe98cd04885757/users/d512d56d7e434a3dbc11c77ce6c69961/roles/e411ad9eb96e4e8d85b59b5b700ada7b` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:19.768300 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116e3013-5ae3-4fbf-bef8-bcb7299d5506 admin admin] RBAC: Authorizing `identity:create_grant(project_id=98cbea710e7043648dbe98cd04885757, user_id=d512d56d7e434a3dbc11c77ce6c69961, role_id=e411ad9eb96e4e8d85b59b5b700ada7b)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:42:19.771018 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116e3013-5ae3-4fbf-bef8-bcb7299d5506 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:42:19.783294 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1111/2228] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 17:42:19 2022] PUT /identity/v3/projects/98cbea710e7043648dbe98cd04885757/users/d512d56d7e434a3dbc11c77ce6c69961/roles/e411ad9eb96e4e8d85b59b5b700ada7b => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:42:20.477935 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfcbcca-7c89-427a-9ed4-77c5608376cd None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:20.478393 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfcbcca-7c89-427a-9ed4-77c5608376cd None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:20.478481 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfcbcca-7c89-427a-9ed4-77c5608376cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:20.508754 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-6dfcbcca-7c89-427a-9ed4-77c5608376cd None None] MFA Rules not processed for user `d512d56d7e434a3dbc11c77ce6c69961`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:42:20.510208 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-6dfcbcca-7c89-427a-9ed4-77c5608376cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:20.560129 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:42:20.599520 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1118/2229] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:42:20 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:42:20.612333 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116e3013-5ae3-4fbf-bef8-bcb7299d5506 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:42:20.614559 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-116e3013-5ae3-4fbf-bef8-bcb7299d5506 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:20.688715 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116e3013-5ae3-4fbf-bef8-bcb7299d5506 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:42:20.692445 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cf337f-ce63-40af-a418-0f638434b76c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59be342541264942ad87f60ff4de6c8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:42:20.693112 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cf337f-ce63-40af-a418-0f638434b76c service cinder] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:20.693391 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cf337f-ce63-40af-a418-0f638434b76c service cinder] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:20.693661 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cf337f-ce63-40af-a418-0f638434b76c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:20.755787 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cf337f-ce63-40af-a418-0f638434b76c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:42:20.757290 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cf337f-ce63-40af-a418-0f638434b76c service cinder] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:42:20.768061 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1112/2230] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:42:20 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:42:27.396430 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfcbcca-7c89-427a-9ed4-77c5608376cd None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:42:27.405783 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfcbcca-7c89-427a-9ed4-77c5608376cd None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:42:27.409260 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6c09e3-cbf0-4085-b14d-30976a2ae3bb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59be342541264942ad87f60ff4de6c8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:42:27.409851 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6c09e3-cbf0-4085-b14d-30976a2ae3bb service cinder] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:27.410126 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6c09e3-cbf0-4085-b14d-30976a2ae3bb service cinder] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:27.410393 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6c09e3-cbf0-4085-b14d-30976a2ae3bb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:27.419624 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6c09e3-cbf0-4085-b14d-30976a2ae3bb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:42:27.421162 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6c09e3-cbf0-4085-b14d-30976a2ae3bb service cinder] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:42:27.430578 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1119/2231] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:42:27 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:42:30.763366 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:INSERT=4,keystone:SELECT=65 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:42:37.426430 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=15 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:42:41.643228 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cf337f-ce63-40af-a418-0f638434b76c service cinder] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:42:41.644712 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-28cf337f-ce63-40af-a418-0f638434b76c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:41.709450 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cf337f-ce63-40af-a418-0f638434b76c service cinder] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:42:41.713967 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea04f70-c647-42a8-8818-005b3bdc1be0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:42:41.714594 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea04f70-c647-42a8-8818-005b3bdc1be0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:41.714875 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea04f70-c647-42a8-8818-005b3bdc1be0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:41.715150 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea04f70-c647-42a8-8818-005b3bdc1be0 admin admin] PATH_INFO: `/v3/users/a31877d157944e71b1ded9856ccd9685` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:41.716253 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea04f70-c647-42a8-8818-005b3bdc1be0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a31877d157944e71b1ded9856ccd9685)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:42:41.718269 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea04f70-c647-42a8-8818-005b3bdc1be0 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:42:41.764356 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-eea04f70-c647-42a8-8818-005b3bdc1be0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a31877d157944e71b1ded9856ccd9685'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:42:41.765045 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-eea04f70-c647-42a8-8818-005b3bdc1be0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a31877d157944e71b1ded9856ccd9685'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:42:41.772828 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-eea04f70-c647-42a8-8818-005b3bdc1be0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a31877d157944e71b1ded9856ccd9685'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:42:41.773499 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-eea04f70-c647-42a8-8818-005b3bdc1be0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a31877d157944e71b1ded9856ccd9685'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:42:41.783498 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-eea04f70-c647-42a8-8818-005b3bdc1be0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a31877d157944e71b1ded9856ccd9685'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:42:41.805552 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1113/2232] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 17:42:41 2022] DELETE /identity/v3/users/a31877d157944e71b1ded9856ccd9685 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:42:41.810055 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2e81c2-0972-4525-8687-d680f21c9541 None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:41.810548 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2e81c2-0972-4525-8687-d680f21c9541 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:41.810643 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2e81c2-0972-4525-8687-d680f21c9541 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:41.831452 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-dd2e81c2-0972-4525-8687-d680f21c9541 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:42:41.832742 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-dd2e81c2-0972-4525-8687-d680f21c9541 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:41.901968 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1120/2233] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:42:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:42:41.917003 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea04f70-c647-42a8-8818-005b3bdc1be0 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:42:41.918424 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-eea04f70-c647-42a8-8818-005b3bdc1be0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:41.974838 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea04f70-c647-42a8-8818-005b3bdc1be0 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:42:41.978458 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dd90e6-7a82-4077-b88d-c322d6dafd41 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43ad98a250a144099ccd18378aedd4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:42:41.979039 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dd90e6-7a82-4077-b88d-c322d6dafd41 service neutron] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:41.979314 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dd90e6-7a82-4077-b88d-c322d6dafd41 service neutron] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:41.979584 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dd90e6-7a82-4077-b88d-c322d6dafd41 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:41.988929 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04dd90e6-7a82-4077-b88d-c322d6dafd41 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:42:41.990355 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04dd90e6-7a82-4077-b88d-c322d6dafd41 service neutron] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:42:41.999555 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1114/2234] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:42:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:42:42.703279 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2e81c2-0972-4525-8687-d680f21c9541 None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:42:42.709842 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-dd2e81c2-0972-4525-8687-d680f21c9541 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:42.780040 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2e81c2-0972-4525-8687-d680f21c9541 None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:42:42.795807 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a156db5-1730-44e5-a848-b780ad262a55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:42:42.796917 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a156db5-1730-44e5-a848-b780ad262a55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:42.797197 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a156db5-1730-44e5-a848-b780ad262a55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:42.798089 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a156db5-1730-44e5-a848-b780ad262a55 admin admin] PATH_INFO: `/v3/projects/d10a1b6bd4364e8c8c0ea743d8601461` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:42.799624 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a156db5-1730-44e5-a848-b780ad262a55 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d10a1b6bd4364e8c8c0ea743d8601461)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:42:42.802410 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a156db5-1730-44e5-a848-b780ad262a55 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:42:42.879532 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-4a156db5-1730-44e5-a848-b780ad262a55 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd10a1b6bd4364e8c8c0ea743d8601461', 'request_id': 'req-4a156db5-1730-44e5-a848-b780ad262a55'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:42:42.888525 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-4a156db5-1730-44e5-a848-b780ad262a55 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd10a1b6bd4364e8c8c0ea743d8601461', 'request_id': 'req-4a156db5-1730-44e5-a848-b780ad262a55'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:42:42.896287 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-4a156db5-1730-44e5-a848-b780ad262a55 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd10a1b6bd4364e8c8c0ea743d8601461', 'request_id': 'req-4a156db5-1730-44e5-a848-b780ad262a55'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:42:42.907384 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-4a156db5-1730-44e5-a848-b780ad262a55 admin admin] The token cache is being invalidate because project d10a1b6bd4364e8c8c0ea743d8601461 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81758) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:42:42.907599 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-4a156db5-1730-44e5-a848-b780ad262a55 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:42:42.908272 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-4a156db5-1730-44e5-a848-b780ad262a55 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:42:42.909768 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1121/2235] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 17:42:42 2022] DELETE /identity/v3/projects/d10a1b6bd4364e8c8c0ea743d8601461 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:42:42.914677 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16aa9f1-dbe8-4949-8bd7-d4d94c1dad45 None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:42.915136 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16aa9f1-dbe8-4949-8bd7-d4d94c1dad45 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:42.915227 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16aa9f1-dbe8-4949-8bd7-d4d94c1dad45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:42.934939 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-c16aa9f1-dbe8-4949-8bd7-d4d94c1dad45 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:42:42.936258 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-c16aa9f1-dbe8-4949-8bd7-d4d94c1dad45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:42.956382 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:42:43.017747 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1115/2236] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:42:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:42:43.026412 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd461c28-cfb4-4212-9526-4ebee4ba05d0 None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:43.026699 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd461c28-cfb4-4212-9526-4ebee4ba05d0 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:43.026976 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd461c28-cfb4-4212-9526-4ebee4ba05d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:43.048033 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-bd461c28-cfb4-4212-9526-4ebee4ba05d0 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:42:43.049456 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-bd461c28-cfb4-4212-9526-4ebee4ba05d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:43.057501 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1122/2237] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:42:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:42:43.061995 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16aa9f1-dbe8-4949-8bd7-d4d94c1dad45 None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:42:43.074094 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16aa9f1-dbe8-4949-8bd7-d4d94c1dad45 None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:42:43.079237 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0db6dd2-4e8c-49e6-84a2-b4775be5fe60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:42:43.080012 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0db6dd2-4e8c-49e6-84a2-b4775be5fe60 admin admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:43.080526 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0db6dd2-4e8c-49e6-84a2-b4775be5fe60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:43.081067 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0db6dd2-4e8c-49e6-84a2-b4775be5fe60 admin admin] PATH_INFO: `/v3/domains` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:43.082421 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0db6dd2-4e8c-49e6-84a2-b4775be5fe60 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:42:43.082832 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0db6dd2-4e8c-49e6-84a2-b4775be5fe60 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:42:43.084663 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0db6dd2-4e8c-49e6-84a2-b4775be5fe60 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:42:43.099143 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1116/2238] 10.222.0.45 () {58 vars in 1153 bytes} [Wed Aug 24 17:42:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:42:43.103669 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd461c28-cfb4-4212-9526-4ebee4ba05d0 None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:42:43.114343 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd461c28-cfb4-4212-9526-4ebee4ba05d0 None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:42:43.119691 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d7a21a-dbe4-4a32-bb15-6ab84337942f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:42:43.120298 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d7a21a-dbe4-4a32-bb15-6ab84337942f admin admin] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:43.120602 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d7a21a-dbe4-4a32-bb15-6ab84337942f admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:43.120880 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d7a21a-dbe4-4a32-bb15-6ab84337942f admin admin] PATH_INFO: `/v3/projects` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:43.121587 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d7a21a-dbe4-4a32-bb15-6ab84337942f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:42:43.123534 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d7a21a-dbe4-4a32-bb15-6ab84337942f admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:42:43.156213 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1123/2239] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 17:42:43 2022] POST /identity/v3/projects => generated 369 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:42:43.162020 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0db6dd2-4e8c-49e6-84a2-b4775be5fe60 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:42:43.173374 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0db6dd2-4e8c-49e6-84a2-b4775be5fe60 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:42:43.230662 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4f8fce-40cf-4b19-b566-32595da07722 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:42:43.231268 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4f8fce-40cf-4b19-b566-32595da07722 admin admin] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:43.231549 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4f8fce-40cf-4b19-b566-32595da07722 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:43.231822 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4f8fce-40cf-4b19-b566-32595da07722 admin admin] PATH_INFO: `/v3/users` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:43.232445 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4f8fce-40cf-4b19-b566-32595da07722 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:42:43.234428 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4f8fce-40cf-4b19-b566-32595da07722 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:42:43.255659 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1117/2240] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 17:42:43 2022] POST /identity/v3/users => generated 285 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:42:43.259806 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d7a21a-dbe4-4a32-bb15-6ab84337942f admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:42:43.270930 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d7a21a-dbe4-4a32-bb15-6ab84337942f admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:42:43.275533 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c88f710-350a-49ea-99b9-f52b8a28f05f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:42:43.276125 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c88f710-350a-49ea-99b9-f52b8a28f05f admin admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:43.276406 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c88f710-350a-49ea-99b9-f52b8a28f05f admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:43.276710 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c88f710-350a-49ea-99b9-f52b8a28f05f admin admin] PATH_INFO: `/v3/roles` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:43.277354 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c88f710-350a-49ea-99b9-f52b8a28f05f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:42:43.278947 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c88f710-350a-49ea-99b9-f52b8a28f05f admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:42:43.289269 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1124/2241] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:42:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 17:42:43.293960 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4f8fce-40cf-4b19-b566-32595da07722 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:42:43.304172 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4f8fce-40cf-4b19-b566-32595da07722 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:42:43.309176 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca17675-2326-499a-88c0-9989340f2792 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:42:43.309777 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca17675-2326-499a-88c0-9989340f2792 admin admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:43.310052 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca17675-2326-499a-88c0-9989340f2792 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:43.310324 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca17675-2326-499a-88c0-9989340f2792 admin admin] PATH_INFO: `/v3/roles` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:43.310956 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca17675-2326-499a-88c0-9989340f2792 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:42:43.312543 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca17675-2326-499a-88c0-9989340f2792 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:42:43.323386 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1118/2242] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:42:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:42:43.327955 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c88f710-350a-49ea-99b9-f52b8a28f05f admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:42:43.338423 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c88f710-350a-49ea-99b9-f52b8a28f05f admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:42:43.342951 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335b22a3-aac0-4156-9b90-d4ae466fcb2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:42:43.344465 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335b22a3-aac0-4156-9b90-d4ae466fcb2a admin admin] REQUEST_METHOD: `PUT` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:43.344784 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335b22a3-aac0-4156-9b90-d4ae466fcb2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:43.345379 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335b22a3-aac0-4156-9b90-d4ae466fcb2a admin admin] PATH_INFO: `/v3/projects/a80f77d1ee7c4226aaf6390614475f65/users/09c6006c56be4e89a4b6d51f3a74f36c/roles/02476a5b82eb4b188c18a5d706a3ef68` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:43.359857 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335b22a3-aac0-4156-9b90-d4ae466fcb2a admin admin] RBAC: Authorizing `identity:create_grant(project_id=a80f77d1ee7c4226aaf6390614475f65, user_id=09c6006c56be4e89a4b6d51f3a74f36c, role_id=02476a5b82eb4b188c18a5d706a3ef68)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:42:43.362531 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335b22a3-aac0-4156-9b90-d4ae466fcb2a admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:42:43.376062 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1125/2243] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 17:42:43 2022] PUT /identity/v3/projects/a80f77d1ee7c4226aaf6390614475f65/users/09c6006c56be4e89a4b6d51f3a74f36c/roles/02476a5b82eb4b188c18a5d706a3ef68 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:42:43.386206 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dd6a45-b7c3-4b92-8b7e-0b8cb87998c1 None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:43.386500 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dd6a45-b7c3-4b92-8b7e-0b8cb87998c1 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:43.386778 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dd6a45-b7c3-4b92-8b7e-0b8cb87998c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:43.414998 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-72dd6a45-b7c3-4b92-8b7e-0b8cb87998c1 None None] MFA Rules not processed for user `09c6006c56be4e89a4b6d51f3a74f36c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:42:43.416222 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-72dd6a45-b7c3-4b92-8b7e-0b8cb87998c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:43.482301 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1119/2244] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:42:43 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:42:43.494359 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335b22a3-aac0-4156-9b90-d4ae466fcb2a admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:42:43.496312 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-335b22a3-aac0-4156-9b90-d4ae466fcb2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:43.519811 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:42:43.559040 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335b22a3-aac0-4156-9b90-d4ae466fcb2a admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:42:43.564598 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c32236-0bdb-4593-9be2-9f8127c3db3d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df5fee57539447428dd1c72eab3cfd9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:42:43.565317 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c32236-0bdb-4593-9be2-9f8127c3db3d service glance] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:43.565516 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c32236-0bdb-4593-9be2-9f8127c3db3d service glance] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:43.565787 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c32236-0bdb-4593-9be2-9f8127c3db3d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:43.574519 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c32236-0bdb-4593-9be2-9f8127c3db3d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:42:43.575978 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c32236-0bdb-4593-9be2-9f8127c3db3d service glance] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:42:43.591520 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1126/2245] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:42:43 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:42:44.542667 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec47a170-07fa-4831-9d4a-82740064cf5a None None] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:44.543138 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec47a170-07fa-4831-9d4a-82740064cf5a None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:44.543337 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec47a170-07fa-4831-9d4a-82740064cf5a None None] PATH_INFO: `/` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:44.544714 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1120/2246] 10.222.0.45 () {54 vars in 921 bytes} [Wed Aug 24 17:42:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:42:44.549601 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0200e0-46f8-4829-b2b8-eddccf9f6004 None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:44.550030 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0200e0-46f8-4829-b2b8-eddccf9f6004 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:44.550279 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0200e0-46f8-4829-b2b8-eddccf9f6004 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:44.570276 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-1e0200e0-46f8-4829-b2b8-eddccf9f6004 None None] MFA Rules not processed for user `37def4721b554b9ebb7704a19642d15b`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:42:44.571490 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-1e0200e0-46f8-4829-b2b8-eddccf9f6004 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:44.652854 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1127/2247] 10.222.0.45 () {58 vars in 1018 bytes} [Wed Aug 24 17:42:44 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:42:44.664696 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec47a170-07fa-4831-9d4a-82740064cf5a None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:42:44.666787 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-ec47a170-07fa-4831-9d4a-82740064cf5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:44.684752 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec47a170-07fa-4831-9d4a-82740064cf5a None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:42:44.689885 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb38a14c-4416-40e9-a8d9-293607c0be7a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37def4721b554b9ebb7704a19642d15b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:42:44.690489 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb38a14c-4416-40e9-a8d9-293607c0be7a service placement] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:44.690765 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb38a14c-4416-40e9-a8d9-293607c0be7a service placement] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:44.691035 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb38a14c-4416-40e9-a8d9-293607c0be7a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:44.699755 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb38a14c-4416-40e9-a8d9-293607c0be7a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:42:44.701390 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb38a14c-4416-40e9-a8d9-293607c0be7a service placement] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:42:44.712946 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1121/2248] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:42:44 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:42:45.230590 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bc4744-de5c-48e0-bcb6-5590f75fa03c None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:45.230887 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bc4744-de5c-48e0-bcb6-5590f75fa03c None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:45.231162 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bc4744-de5c-48e0-bcb6-5590f75fa03c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:45.257265 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-d0bc4744-de5c-48e0-bcb6-5590f75fa03c None None] MFA Rules not processed for user `ea143d042ee94ae7a4544d90d18db7db`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:42:45.258682 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-d0bc4744-de5c-48e0-bcb6-5590f75fa03c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:45.328978 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1128/2249] 10.222.0.45 () {58 vars in 1009 bytes} [Wed Aug 24 17:42:45 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:42:45.337419 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb38a14c-4416-40e9-a8d9-293607c0be7a service placement] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:42:45.338894 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-fb38a14c-4416-40e9-a8d9-293607c0be7a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:45.346194 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:42:45.396295 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bc4744-de5c-48e0-bcb6-5590f75fa03c None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:42:45.397867 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-d0bc4744-de5c-48e0-bcb6-5590f75fa03c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:45.417426 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb38a14c-4416-40e9-a8d9-293607c0be7a service placement] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:42:45.423908 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03897bf3-0352-4403-8a2a-912695e7af4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:42:45.424706 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03897bf3-0352-4403-8a2a-912695e7af4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:45.425201 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03897bf3-0352-4403-8a2a-912695e7af4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:45.425550 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03897bf3-0352-4403-8a2a-912695e7af4b admin admin] PATH_INFO: `/v3/users/5d9926174a044ba4a36960c2c64ec5a9` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:45.426823 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03897bf3-0352-4403-8a2a-912695e7af4b admin admin] RBAC: Authorizing `identity:delete_user(user_id=5d9926174a044ba4a36960c2c64ec5a9)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:42:45.429628 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03897bf3-0352-4403-8a2a-912695e7af4b admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:42:45.487580 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bc4744-de5c-48e0-bcb6-5590f75fa03c None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:42:45.492617 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be1e1ef-c986-44c4-8e72-1383ea41fcad service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90f0772b64c84faaa689e7272fa97898', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:42:45.493225 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be1e1ef-c986-44c4-8e72-1383ea41fcad service swift] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:45.493503 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be1e1ef-c986-44c4-8e72-1383ea41fcad service swift] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:45.493773 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be1e1ef-c986-44c4-8e72-1383ea41fcad service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:45.495578 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-03897bf3-0352-4403-8a2a-912695e7af4b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5d9926174a044ba4a36960c2c64ec5a9'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:42:45.496279 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-03897bf3-0352-4403-8a2a-912695e7af4b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5d9926174a044ba4a36960c2c64ec5a9'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:42:45.498599 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:42:45.503914 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be1e1ef-c986-44c4-8e72-1383ea41fcad service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:42:45.504047 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-03897bf3-0352-4403-8a2a-912695e7af4b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5d9926174a044ba4a36960c2c64ec5a9'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:42:45.504850 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-03897bf3-0352-4403-8a2a-912695e7af4b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5d9926174a044ba4a36960c2c64ec5a9'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:42:45.505534 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be1e1ef-c986-44c4-8e72-1383ea41fcad service swift] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:42:45.512276 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-03897bf3-0352-4403-8a2a-912695e7af4b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5d9926174a044ba4a36960c2c64ec5a9'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:42:45.514538 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1129/2250] 10.222.0.45 () {60 vars in 1384 bytes} [Wed Aug 24 17:42:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 24 17:42:45.545267 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1122/2251] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 17:42:45 2022] DELETE /identity/v3/users/5d9926174a044ba4a36960c2c64ec5a9 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:42:45.869427 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be1e1ef-c986-44c4-8e72-1383ea41fcad service swift] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:42:45.884361 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be1e1ef-c986-44c4-8e72-1383ea41fcad service swift] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:42:45.963839 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5085c27-c74b-44d8-b5eb-c301716d1ad0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:42:45.964619 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5085c27-c74b-44d8-b5eb-c301716d1ad0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:45.965008 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5085c27-c74b-44d8-b5eb-c301716d1ad0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:45.965385 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5085c27-c74b-44d8-b5eb-c301716d1ad0 admin admin] PATH_INFO: `/v3/projects/04d2d2cf5a2849a88cbfa9885ee2d492` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:45.966637 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5085c27-c74b-44d8-b5eb-c301716d1ad0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=04d2d2cf5a2849a88cbfa9885ee2d492)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:42:45.969310 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5085c27-c74b-44d8-b5eb-c301716d1ad0 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:42:46.037605 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-e5085c27-c74b-44d8-b5eb-c301716d1ad0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '04d2d2cf5a2849a88cbfa9885ee2d492', 'request_id': 'req-e5085c27-c74b-44d8-b5eb-c301716d1ad0'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:42:46.049161 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-e5085c27-c74b-44d8-b5eb-c301716d1ad0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '04d2d2cf5a2849a88cbfa9885ee2d492', 'request_id': 'req-e5085c27-c74b-44d8-b5eb-c301716d1ad0'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:42:46.056335 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-e5085c27-c74b-44d8-b5eb-c301716d1ad0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '04d2d2cf5a2849a88cbfa9885ee2d492', 'request_id': 'req-e5085c27-c74b-44d8-b5eb-c301716d1ad0'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:42:46.069074 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-e5085c27-c74b-44d8-b5eb-c301716d1ad0 admin admin] The token cache is being invalidate because project 04d2d2cf5a2849a88cbfa9885ee2d492 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81758) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:42:46.070112 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-e5085c27-c74b-44d8-b5eb-c301716d1ad0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:42:46.070865 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-e5085c27-c74b-44d8-b5eb-c301716d1ad0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:42:46.077126 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1130/2252] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 17:42:45 2022] DELETE /identity/v3/projects/04d2d2cf5a2849a88cbfa9885ee2d492 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:42:46.079139 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03897bf3-0352-4403-8a2a-912695e7af4b admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:42:46.082296 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-03897bf3-0352-4403-8a2a-912695e7af4b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:46.098492 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03897bf3-0352-4403-8a2a-912695e7af4b admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:42:46.140789 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bf154e-8443-418a-9a88-13c682873428 None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:46.141083 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bf154e-8443-418a-9a88-13c682873428 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:46.141758 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bf154e-8443-418a-9a88-13c682873428 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:46.168001 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-01bf154e-8443-418a-9a88-13c682873428 None None] MFA Rules not processed for user `ea143d042ee94ae7a4544d90d18db7db`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:42:46.168290 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbcbf93-4f4f-4b97-9d33-f2277c16e7dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:42:46.168985 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbcbf93-4f4f-4b97-9d33-f2277c16e7dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:46.169266 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbcbf93-4f4f-4b97-9d33-f2277c16e7dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:46.169544 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbcbf93-4f4f-4b97-9d33-f2277c16e7dc admin admin] PATH_INFO: `/v3/users/d512d56d7e434a3dbc11c77ce6c69961` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:46.170722 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbcbf93-4f4f-4b97-9d33-f2277c16e7dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=d512d56d7e434a3dbc11c77ce6c69961)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:42:46.172670 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-01bf154e-8443-418a-9a88-13c682873428 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:46.172952 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbcbf93-4f4f-4b97-9d33-f2277c16e7dc admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:42:46.225577 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:42:46.258747 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1131/2253] 10.222.0.45 () {58 vars in 967 bytes} [Wed Aug 24 17:42:46 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:42:46.268968 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-3fbcbf93-4f4f-4b97-9d33-f2277c16e7dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd512d56d7e434a3dbc11c77ce6c69961'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:42:46.269888 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-3fbcbf93-4f4f-4b97-9d33-f2277c16e7dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd512d56d7e434a3dbc11c77ce6c69961'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:42:46.283253 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bf154e-8443-418a-9a88-13c682873428 None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:42:46.285719 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-01bf154e-8443-418a-9a88-13c682873428 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:46.301271 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-3fbcbf93-4f4f-4b97-9d33-f2277c16e7dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd512d56d7e434a3dbc11c77ce6c69961'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:42:46.302980 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-3fbcbf93-4f4f-4b97-9d33-f2277c16e7dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd512d56d7e434a3dbc11c77ce6c69961'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:42:46.319694 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-3fbcbf93-4f4f-4b97-9d33-f2277c16e7dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd512d56d7e434a3dbc11c77ce6c69961'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:42:46.329395 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:42:46.359920 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1123/2254] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 17:42:46 2022] DELETE /identity/v3/users/d512d56d7e434a3dbc11c77ce6c69961 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:42:46.409055 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bf154e-8443-418a-9a88-13c682873428 None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:42:46.509606 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5c408b-378f-4964-8b9e-f761fd877939 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90f0772b64c84faaa689e7272fa97898', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:42:46.510168 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5c408b-378f-4964-8b9e-f761fd877939 service swift] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:46.510453 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5c408b-378f-4964-8b9e-f761fd877939 service swift] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:46.510906 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5c408b-378f-4964-8b9e-f761fd877939 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:46.513619 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-4a5c408b-378f-4964-8b9e-f761fd877939 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:46.559746 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbcbf93-4f4f-4b97-9d33-f2277c16e7dc admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:42:46.561328 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-3fbcbf93-4f4f-4b97-9d33-f2277c16e7dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:46.634641 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5c408b-378f-4964-8b9e-f761fd877939 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:42:46.641180 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5c408b-378f-4964-8b9e-f761fd877939 service swift] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:42:46.653068 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1132/2255] 10.222.0.45 () {60 vars in 1384 bytes} [Wed Aug 24 17:42:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 17:42:46.665201 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbcbf93-4f4f-4b97-9d33-f2277c16e7dc admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:42:46.675131 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7931ca63-e9a7-495f-a0bc-393f080d703c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:42:46.676674 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7931ca63-e9a7-495f-a0bc-393f080d703c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:46.677059 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7931ca63-e9a7-495f-a0bc-393f080d703c admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:46.678573 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7931ca63-e9a7-495f-a0bc-393f080d703c admin admin] PATH_INFO: `/v3/projects/98cbea710e7043648dbe98cd04885757` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:46.680842 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7931ca63-e9a7-495f-a0bc-393f080d703c admin admin] RBAC: Authorizing `identity:delete_project(project_id=98cbea710e7043648dbe98cd04885757)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:42:46.682902 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7931ca63-e9a7-495f-a0bc-393f080d703c admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:42:46.782656 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-7931ca63-e9a7-495f-a0bc-393f080d703c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '98cbea710e7043648dbe98cd04885757', 'request_id': 'req-7931ca63-e9a7-495f-a0bc-393f080d703c'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:42:46.795274 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-7931ca63-e9a7-495f-a0bc-393f080d703c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '98cbea710e7043648dbe98cd04885757', 'request_id': 'req-7931ca63-e9a7-495f-a0bc-393f080d703c'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:42:46.806308 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-7931ca63-e9a7-495f-a0bc-393f080d703c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '98cbea710e7043648dbe98cd04885757', 'request_id': 'req-7931ca63-e9a7-495f-a0bc-393f080d703c'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:42:46.829931 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-7931ca63-e9a7-495f-a0bc-393f080d703c admin admin] The token cache is being invalidate because project 98cbea710e7043648dbe98cd04885757 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81759) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:42:46.830241 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-7931ca63-e9a7-495f-a0bc-393f080d703c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:42:46.830975 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-7931ca63-e9a7-495f-a0bc-393f080d703c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:42:46.834518 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1124/2256] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 17:42:46 2022] DELETE /identity/v3/projects/98cbea710e7043648dbe98cd04885757 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:42:46.839536 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd91a3e-a086-492b-a238-9c480eb64461 None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:46.839851 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd91a3e-a086-492b-a238-9c480eb64461 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:46.840098 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd91a3e-a086-492b-a238-9c480eb64461 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:46.886714 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-8cd91a3e-a086-492b-a238-9c480eb64461 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:42:46.889104 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-8cd91a3e-a086-492b-a238-9c480eb64461 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:46.964080 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1133/2257] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:42:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:42:46.971691 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46953158-f13c-4aea-b813-276947d4975c None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:46.971983 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46953158-f13c-4aea-b813-276947d4975c None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:46.972266 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46953158-f13c-4aea-b813-276947d4975c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:46.993333 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-46953158-f13c-4aea-b813-276947d4975c None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:42:46.994727 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-46953158-f13c-4aea-b813-276947d4975c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:47.003936 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1125/2258] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:42:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:42:47.008759 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd91a3e-a086-492b-a238-9c480eb64461 None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:42:47.022958 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd91a3e-a086-492b-a238-9c480eb64461 None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:42:47.027854 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985cbe5b-1a21-4bd5-95ce-278ffee5748a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:42:47.028460 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985cbe5b-1a21-4bd5-95ce-278ffee5748a admin admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:47.028693 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7da456-1706-4364-aa60-e5431fee9b2c None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:47.028799 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985cbe5b-1a21-4bd5-95ce-278ffee5748a admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:47.028962 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7da456-1706-4364-aa60-e5431fee9b2c None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:47.029084 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985cbe5b-1a21-4bd5-95ce-278ffee5748a admin admin] PATH_INFO: `/v3/domains` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:47.029307 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7da456-1706-4364-aa60-e5431fee9b2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:47.029902 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985cbe5b-1a21-4bd5-95ce-278ffee5748a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:42:47.030205 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985cbe5b-1a21-4bd5-95ce-278ffee5748a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:42:47.031837 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985cbe5b-1a21-4bd5-95ce-278ffee5748a admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:42:47.042054 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:42:47.045302 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1134/2259] 10.222.0.45 () {58 vars in 1153 bytes} [Wed Aug 24 17:42:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:42:47.050861 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985cbe5b-1a21-4bd5-95ce-278ffee5748a admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:42:47.054902 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-6b7da456-1706-4364-aa60-e5431fee9b2c None None] MFA Rules not processed for user `ea143d042ee94ae7a4544d90d18db7db`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:42:47.056775 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-6b7da456-1706-4364-aa60-e5431fee9b2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:47.065783 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985cbe5b-1a21-4bd5-95ce-278ffee5748a admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:42:47.070904 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772f7785-c46d-492c-88a7-8352fe1f9617 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:42:47.071512 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772f7785-c46d-492c-88a7-8352fe1f9617 admin admin] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:47.071915 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772f7785-c46d-492c-88a7-8352fe1f9617 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:47.072199 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772f7785-c46d-492c-88a7-8352fe1f9617 admin admin] PATH_INFO: `/v3/projects` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:47.072902 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772f7785-c46d-492c-88a7-8352fe1f9617 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:42:47.075252 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772f7785-c46d-492c-88a7-8352fe1f9617 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:42:47.082350 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:42:47.117212 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1135/2260] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 17:42:47 2022] POST /identity/v3/projects => generated 383 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:42:47.123572 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772f7785-c46d-492c-88a7-8352fe1f9617 admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:42:47.150365 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1126/2261] 10.222.0.45 () {58 vars in 1009 bytes} [Wed Aug 24 17:42:47 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:42:47.166189 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7da456-1706-4364-aa60-e5431fee9b2c None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:42:47.168666 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-6b7da456-1706-4364-aa60-e5431fee9b2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:47.202882 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772f7785-c46d-492c-88a7-8352fe1f9617 admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:42:47.207859 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2e3f66-c05c-4021-855f-3bd02fdf0654 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:42:47.208491 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2e3f66-c05c-4021-855f-3bd02fdf0654 admin admin] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:47.208788 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2e3f66-c05c-4021-855f-3bd02fdf0654 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:47.209111 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2e3f66-c05c-4021-855f-3bd02fdf0654 admin admin] PATH_INFO: `/v3/users` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:47.209744 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2e3f66-c05c-4021-855f-3bd02fdf0654 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:42:47.215617 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2e3f66-c05c-4021-855f-3bd02fdf0654 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:42:47.245187 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1136/2262] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 17:42:47 2022] POST /identity/v3/users => generated 292 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:42:47.246146 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2e3f66-c05c-4021-855f-3bd02fdf0654 admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:42:47.258422 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2e3f66-c05c-4021-855f-3bd02fdf0654 admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:42:47.264737 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7da456-1706-4364-aa60-e5431fee9b2c None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:42:47.265037 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b800cf-d57d-4b44-a867-d9413233e5e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:42:47.265769 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b800cf-d57d-4b44-a867-d9413233e5e5 admin admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:47.266074 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b800cf-d57d-4b44-a867-d9413233e5e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:47.266718 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b800cf-d57d-4b44-a867-d9413233e5e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:47.267382 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b800cf-d57d-4b44-a867-d9413233e5e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:42:47.269012 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b800cf-d57d-4b44-a867-d9413233e5e5 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:42:47.280516 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1137/2263] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:42:47 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:42:47.288825 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b800cf-d57d-4b44-a867-d9413233e5e5 admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:42:47.308105 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b800cf-d57d-4b44-a867-d9413233e5e5 admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:42:47.316507 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5804405b-c457-45b0-8b94-bb5dcee6a36f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:42:47.317161 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5804405b-c457-45b0-8b94-bb5dcee6a36f admin admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:47.317440 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5804405b-c457-45b0-8b94-bb5dcee6a36f admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:47.317745 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5804405b-c457-45b0-8b94-bb5dcee6a36f admin admin] PATH_INFO: `/v3/roles` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:47.318407 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5804405b-c457-45b0-8b94-bb5dcee6a36f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:42:47.320030 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5804405b-c457-45b0-8b94-bb5dcee6a36f admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:42:47.336728 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1138/2264] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:42:47 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:42:47.341551 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5804405b-c457-45b0-8b94-bb5dcee6a36f admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:42:47.359924 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5804405b-c457-45b0-8b94-bb5dcee6a36f admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:42:47.368523 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616dd67c-b70c-4840-92d4-4143f719bbf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:42:47.369046 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9aa3ce-3ca2-4f22-bf13-56add33268b0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90f0772b64c84faaa689e7272fa97898', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:42:47.369842 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616dd67c-b70c-4840-92d4-4143f719bbf2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:47.369972 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9aa3ce-3ca2-4f22-bf13-56add33268b0 service swift] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:47.370147 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9aa3ce-3ca2-4f22-bf13-56add33268b0 service swift] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:47.370398 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9aa3ce-3ca2-4f22-bf13-56add33268b0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:47.370846 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616dd67c-b70c-4840-92d4-4143f719bbf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:47.371332 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616dd67c-b70c-4840-92d4-4143f719bbf2 admin admin] PATH_INFO: `/v3/projects/682a59c0b5594a9e910dfdf6179ffa4d/users/80622be871294a6f925d6998ef6c2653/roles/02476a5b82eb4b188c18a5d706a3ef68` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:47.387944 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616dd67c-b70c-4840-92d4-4143f719bbf2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=682a59c0b5594a9e910dfdf6179ffa4d, user_id=80622be871294a6f925d6998ef6c2653, role_id=02476a5b82eb4b188c18a5d706a3ef68)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:42:47.390735 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616dd67c-b70c-4840-92d4-4143f719bbf2 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:42:47.429541 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1139/2265] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 17:42:47 2022] PUT /identity/v3/projects/682a59c0b5594a9e910dfdf6179ffa4d/users/80622be871294a6f925d6998ef6c2653/roles/02476a5b82eb4b188c18a5d706a3ef68 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:42:47.433255 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5853268-00c4-47ea-b8bb-0ec1bfbc98a8 None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:47.433683 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5853268-00c4-47ea-b8bb-0ec1bfbc98a8 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:47.434062 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5853268-00c4-47ea-b8bb-0ec1bfbc98a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:47.462010 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9aa3ce-3ca2-4f22-bf13-56add33268b0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:42:47.463693 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9aa3ce-3ca2-4f22-bf13-56add33268b0 service swift] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:42:47.468907 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-b5853268-00c4-47ea-b8bb-0ec1bfbc98a8 None None] MFA Rules not processed for user `80622be871294a6f925d6998ef6c2653`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:42:47.470380 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-b5853268-00c4-47ea-b8bb-0ec1bfbc98a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:47.475398 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1127/2266] 10.222.0.45 () {60 vars in 1384 bytes} [Wed Aug 24 17:42:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 17:42:47.560592 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1140/2267] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:42:47 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:42:47.573572 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9aa3ce-3ca2-4f22-bf13-56add33268b0 service swift] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:42:47.577376 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-6e9aa3ce-3ca2-4f22-bf13-56add33268b0 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:47.598832 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:42:47.642937 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9aa3ce-3ca2-4f22-bf13-56add33268b0 service swift] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:42:47.695298 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1270ffd-0871-4ff4-b518-120298fac671 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59be342541264942ad87f60ff4de6c8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:42:47.696073 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1270ffd-0871-4ff4-b518-120298fac671 service cinder] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:47.696211 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1270ffd-0871-4ff4-b518-120298fac671 service cinder] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:47.696437 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1270ffd-0871-4ff4-b518-120298fac671 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:47.707895 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1270ffd-0871-4ff4-b518-120298fac671 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:42:47.709371 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1270ffd-0871-4ff4-b518-120298fac671 service cinder] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:42:47.719823 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1128/2268] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:42:47 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:42:47.738910 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5853268-00c4-47ea-b8bb-0ec1bfbc98a8 None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:42:47.751453 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:42:47.831628 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5853268-00c4-47ea-b8bb-0ec1bfbc98a8 None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:42:47.837216 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4670d721-977d-4bfe-af36-0424fca17817 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:42:47.838040 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4670d721-977d-4bfe-af36-0424fca17817 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:47.838464 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4670d721-977d-4bfe-af36-0424fca17817 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:47.838892 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4670d721-977d-4bfe-af36-0424fca17817 admin admin] PATH_INFO: `/v3/users/80622be871294a6f925d6998ef6c2653` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:47.840609 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4670d721-977d-4bfe-af36-0424fca17817 admin admin] RBAC: Authorizing `identity:delete_user(user_id=80622be871294a6f925d6998ef6c2653)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:42:47.842640 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4670d721-977d-4bfe-af36-0424fca17817 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:42:47.875753 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c7af8d-e75f-4aa7-908b-cc79c79ebc62 None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:47.876042 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c7af8d-e75f-4aa7-908b-cc79c79ebc62 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:47.876327 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c7af8d-e75f-4aa7-908b-cc79c79ebc62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:47.897488 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-74c7af8d-e75f-4aa7-908b-cc79c79ebc62 None None] MFA Rules not processed for user `ea143d042ee94ae7a4544d90d18db7db`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:42:47.898167 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-4670d721-977d-4bfe-af36-0424fca17817 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '80622be871294a6f925d6998ef6c2653'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:42:47.898826 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-4670d721-977d-4bfe-af36-0424fca17817 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '80622be871294a6f925d6998ef6c2653'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:42:47.898965 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-74c7af8d-e75f-4aa7-908b-cc79c79ebc62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:47.907434 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-4670d721-977d-4bfe-af36-0424fca17817 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '80622be871294a6f925d6998ef6c2653'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:42:47.908328 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-4670d721-977d-4bfe-af36-0424fca17817 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '80622be871294a6f925d6998ef6c2653'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:42:47.927216 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-4670d721-977d-4bfe-af36-0424fca17817 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '80622be871294a6f925d6998ef6c2653'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:42:47.943644 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1141/2269] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 17:42:47 2022] DELETE /identity/v3/users/80622be871294a6f925d6998ef6c2653 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:42:47.951868 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a9e142-5a6a-45d1-a013-f9adcb0f5792 None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:47.956303 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a9e142-5a6a-45d1-a013-f9adcb0f5792 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:47.956303 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a9e142-5a6a-45d1-a013-f9adcb0f5792 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:47.970339 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1129/2270] 10.222.0.45 () {58 vars in 967 bytes} [Wed Aug 24 17:42:47 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:42:47.980996 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-38a9e142-5a6a-45d1-a013-f9adcb0f5792 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:42:47.982504 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-38a9e142-5a6a-45d1-a013-f9adcb0f5792 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:47.982933 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c7af8d-e75f-4aa7-908b-cc79c79ebc62 None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:42:47.984609 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-74c7af8d-e75f-4aa7-908b-cc79c79ebc62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:48.066238 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1142/2271] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:42:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:42:48.068935 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c7af8d-e75f-4aa7-908b-cc79c79ebc62 None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:42:48.076629 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a9e142-5a6a-45d1-a013-f9adcb0f5792 None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:42:48.078586 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-38a9e142-5a6a-45d1-a013-f9adcb0f5792 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:48.088541 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:42:48.114359 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:42:48.131789 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c9f7cc-6fb2-4f1b-b113-e0e24f930f24 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90f0772b64c84faaa689e7272fa97898', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:42:48.132332 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c9f7cc-6fb2-4f1b-b113-e0e24f930f24 service swift] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:48.132852 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c9f7cc-6fb2-4f1b-b113-e0e24f930f24 service swift] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:48.133131 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c9f7cc-6fb2-4f1b-b113-e0e24f930f24 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:48.142474 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a9e142-5a6a-45d1-a013-f9adcb0f5792 None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:42:48.146156 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f72b601-8a3f-41a9-899d-bb088ec5b3cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43ad98a250a144099ccd18378aedd4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:42:48.146780 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f72b601-8a3f-41a9-899d-bb088ec5b3cf service neutron] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:48.147040 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f72b601-8a3f-41a9-899d-bb088ec5b3cf service neutron] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:48.147315 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f72b601-8a3f-41a9-899d-bb088ec5b3cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:48.156357 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f72b601-8a3f-41a9-899d-bb088ec5b3cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:42:48.157822 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f72b601-8a3f-41a9-899d-bb088ec5b3cf service neutron] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:42:48.167212 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1143/2272] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:42:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 17:42:48.190864 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c9f7cc-6fb2-4f1b-b113-e0e24f930f24 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:42:48.192482 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c9f7cc-6fb2-4f1b-b113-e0e24f930f24 service swift] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:42:48.201166 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1130/2273] 10.222.0.45 () {60 vars in 1384 bytes} [Wed Aug 24 17:42:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 24 17:42:48.483409 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f72b601-8a3f-41a9-899d-bb088ec5b3cf service neutron] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:42:48.485682 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-1f72b601-8a3f-41a9-899d-bb088ec5b3cf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:48.512247 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f72b601-8a3f-41a9-899d-bb088ec5b3cf service neutron] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:42:48.520929 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a17f77f-5aa7-4411-be7f-7e2710480032 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:42:48.521816 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a17f77f-5aa7-4411-be7f-7e2710480032 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:48.522163 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a17f77f-5aa7-4411-be7f-7e2710480032 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:48.522506 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a17f77f-5aa7-4411-be7f-7e2710480032 admin admin] PATH_INFO: `/v3/projects/682a59c0b5594a9e910dfdf6179ffa4d` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:48.524035 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a17f77f-5aa7-4411-be7f-7e2710480032 admin admin] RBAC: Authorizing `identity:delete_project(project_id=682a59c0b5594a9e910dfdf6179ffa4d)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:42:48.526327 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a17f77f-5aa7-4411-be7f-7e2710480032 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:42:48.595946 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-4a17f77f-5aa7-4411-be7f-7e2710480032 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '682a59c0b5594a9e910dfdf6179ffa4d', 'request_id': 'req-4a17f77f-5aa7-4411-be7f-7e2710480032'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:42:48.605367 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-4a17f77f-5aa7-4411-be7f-7e2710480032 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '682a59c0b5594a9e910dfdf6179ffa4d', 'request_id': 'req-4a17f77f-5aa7-4411-be7f-7e2710480032'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:42:48.614673 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-4a17f77f-5aa7-4411-be7f-7e2710480032 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '682a59c0b5594a9e910dfdf6179ffa4d', 'request_id': 'req-4a17f77f-5aa7-4411-be7f-7e2710480032'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:42:48.627857 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-4a17f77f-5aa7-4411-be7f-7e2710480032 admin admin] The token cache is being invalidate because project 682a59c0b5594a9e910dfdf6179ffa4d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81758) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:42:48.628223 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-4a17f77f-5aa7-4411-be7f-7e2710480032 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:42:48.628962 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-4a17f77f-5aa7-4411-be7f-7e2710480032 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:42:48.630584 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1144/2274] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 17:42:48 2022] DELETE /identity/v3/projects/682a59c0b5594a9e910dfdf6179ffa4d => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:42:48.635521 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d79109-5565-47f7-bc01-e30e5c35f597 None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:48.636037 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d79109-5565-47f7-bc01-e30e5c35f597 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:48.636207 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d79109-5565-47f7-bc01-e30e5c35f597 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:48.661960 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-15d79109-5565-47f7-bc01-e30e5c35f597 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:42:48.664945 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-15d79109-5565-47f7-bc01-e30e5c35f597 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:48.733123 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1131/2275] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:42:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:42:48.739793 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e77307-1c33-4a38-a95c-05136ebc92c1 None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:48.740090 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e77307-1c33-4a38-a95c-05136ebc92c1 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:48.740355 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e77307-1c33-4a38-a95c-05136ebc92c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:48.767037 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-00e77307-1c33-4a38-a95c-05136ebc92c1 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:42:48.768261 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-00e77307-1c33-4a38-a95c-05136ebc92c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:48.784609 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1145/2276] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:42:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:42:48.788676 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d79109-5565-47f7-bc01-e30e5c35f597 None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:42:48.798913 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d79109-5565-47f7-bc01-e30e5c35f597 None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:42:48.804929 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b918ad6-5e00-4f42-821f-5a38f95f59f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:42:48.805552 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b918ad6-5e00-4f42-821f-5a38f95f59f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:48.805845 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b918ad6-5e00-4f42-821f-5a38f95f59f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:48.806122 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b918ad6-5e00-4f42-821f-5a38f95f59f2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:48.806788 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b918ad6-5e00-4f42-821f-5a38f95f59f2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:42:48.807085 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b918ad6-5e00-4f42-821f-5a38f95f59f2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:42:48.809110 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b918ad6-5e00-4f42-821f-5a38f95f59f2 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:42:48.823434 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1132/2277] 10.222.0.45 () {58 vars in 1153 bytes} [Wed Aug 24 17:42:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:42:48.828098 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e77307-1c33-4a38-a95c-05136ebc92c1 None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:42:48.841704 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e77307-1c33-4a38-a95c-05136ebc92c1 None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:42:48.847652 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e82f9a-8b50-4974-b70a-190b81c8f747 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:42:48.848268 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e82f9a-8b50-4974-b70a-190b81c8f747 admin admin] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:48.848572 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e82f9a-8b50-4974-b70a-190b81c8f747 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:48.848874 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e82f9a-8b50-4974-b70a-190b81c8f747 admin admin] PATH_INFO: `/v3/projects` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:48.849569 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e82f9a-8b50-4974-b70a-190b81c8f747 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:42:48.851553 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e82f9a-8b50-4974-b70a-190b81c8f747 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:42:48.898236 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1146/2278] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 17:42:48 2022] POST /identity/v3/projects => generated 377 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:42:48.899546 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b918ad6-5e00-4f42-821f-5a38f95f59f2 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:42:48.935956 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b918ad6-5e00-4f42-821f-5a38f95f59f2 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:42:49.048951 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:42:49.068941 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7260dbc1-da62-4a60-b425-cc36782b094f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:42:49.069568 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7260dbc1-da62-4a60-b425-cc36782b094f admin admin] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:49.069879 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7260dbc1-da62-4a60-b425-cc36782b094f admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:49.070171 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7260dbc1-da62-4a60-b425-cc36782b094f admin admin] PATH_INFO: `/v3/users` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:49.070817 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7260dbc1-da62-4a60-b425-cc36782b094f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:42:49.072774 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7260dbc1-da62-4a60-b425-cc36782b094f admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:42:49.111968 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1133/2279] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 17:42:48 2022] POST /identity/v3/users => generated 289 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:42:49.115365 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e82f9a-8b50-4974-b70a-190b81c8f747 admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:42:49.139783 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e82f9a-8b50-4974-b70a-190b81c8f747 admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:42:49.160639 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0b9a94-992f-4f0c-8a51-13e746ba1f32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:42:49.160639 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0b9a94-992f-4f0c-8a51-13e746ba1f32 admin admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:49.160639 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0b9a94-992f-4f0c-8a51-13e746ba1f32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:49.160639 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0b9a94-992f-4f0c-8a51-13e746ba1f32 admin admin] PATH_INFO: `/v3/roles` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:49.164701 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0b9a94-992f-4f0c-8a51-13e746ba1f32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:42:49.167262 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0b9a94-992f-4f0c-8a51-13e746ba1f32 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:42:49.181266 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1147/2280] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:42:49 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:42:49.185516 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7260dbc1-da62-4a60-b425-cc36782b094f admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:42:49.198013 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7260dbc1-da62-4a60-b425-cc36782b094f admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:42:49.204410 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5922f1-a2d0-43be-9e3e-a2f5f94cf214 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:42:49.205192 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5922f1-a2d0-43be-9e3e-a2f5f94cf214 admin admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:49.205612 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5922f1-a2d0-43be-9e3e-a2f5f94cf214 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:49.206053 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5922f1-a2d0-43be-9e3e-a2f5f94cf214 admin admin] PATH_INFO: `/v3/roles` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:49.206885 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5922f1-a2d0-43be-9e3e-a2f5f94cf214 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:42:49.211270 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5922f1-a2d0-43be-9e3e-a2f5f94cf214 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:42:49.245331 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1134/2281] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:42:49 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:42:49.250525 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0b9a94-992f-4f0c-8a51-13e746ba1f32 admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:42:49.263785 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0b9a94-992f-4f0c-8a51-13e746ba1f32 admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:42:49.282287 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33726409-db37-48d2-8860-2040282d43f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:42:49.283171 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33726409-db37-48d2-8860-2040282d43f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:49.283634 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33726409-db37-48d2-8860-2040282d43f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:49.284063 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33726409-db37-48d2-8860-2040282d43f6 admin admin] PATH_INFO: `/v3/projects/b0643edf95e84451b9ffd8fc79891803/users/6e6ef8b7c265474f888d7b78fb9e2bb7/roles/02476a5b82eb4b188c18a5d706a3ef68` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:49.301637 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33726409-db37-48d2-8860-2040282d43f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0643edf95e84451b9ffd8fc79891803, user_id=6e6ef8b7c265474f888d7b78fb9e2bb7, role_id=02476a5b82eb4b188c18a5d706a3ef68)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:42:49.304325 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33726409-db37-48d2-8860-2040282d43f6 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:42:49.320865 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1148/2282] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 17:42:49 2022] PUT /identity/v3/projects/b0643edf95e84451b9ffd8fc79891803/users/6e6ef8b7c265474f888d7b78fb9e2bb7/roles/02476a5b82eb4b188c18a5d706a3ef68 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:42:49.330805 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c764c9ac-722f-4de8-8a0f-eb9eae333fb1 None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:49.331086 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c764c9ac-722f-4de8-8a0f-eb9eae333fb1 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:49.331378 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c764c9ac-722f-4de8-8a0f-eb9eae333fb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:49.366899 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-c764c9ac-722f-4de8-8a0f-eb9eae333fb1 None None] MFA Rules not processed for user `6e6ef8b7c265474f888d7b78fb9e2bb7`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:42:49.368129 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-c764c9ac-722f-4de8-8a0f-eb9eae333fb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:49.465338 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1135/2283] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:42:49 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:42:49.470269 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33726409-db37-48d2-8860-2040282d43f6 admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:42:49.557106 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:42:49.568258 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33726409-db37-48d2-8860-2040282d43f6 admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:42:49.573965 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bab901-28a0-418d-aa14-55851e260156 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:42:49.574666 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bab901-28a0-418d-aa14-55851e260156 admin admin] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:49.574895 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bab901-28a0-418d-aa14-55851e260156 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:49.575199 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bab901-28a0-418d-aa14-55851e260156 admin admin] PATH_INFO: `/v3/projects` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:49.575838 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bab901-28a0-418d-aa14-55851e260156 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:42:49.577901 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bab901-28a0-418d-aa14-55851e260156 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:42:49.605466 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1149/2284] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 17:42:49 2022] POST /identity/v3/projects => generated 375 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:42:49.610478 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c764c9ac-722f-4de8-8a0f-eb9eae333fb1 None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:42:49.621346 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c764c9ac-722f-4de8-8a0f-eb9eae333fb1 None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:42:49.680109 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b08edaa-8573-4179-9e45-e85d961d775c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:42:49.680875 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b08edaa-8573-4179-9e45-e85d961d775c admin admin] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:49.680998 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b08edaa-8573-4179-9e45-e85d961d775c admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:49.681260 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b08edaa-8573-4179-9e45-e85d961d775c admin admin] PATH_INFO: `/v3/users` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:49.681869 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b08edaa-8573-4179-9e45-e85d961d775c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:42:49.684069 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b08edaa-8573-4179-9e45-e85d961d775c admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:42:49.709660 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1136/2285] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 17:42:49 2022] POST /identity/v3/users => generated 288 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:42:49.713551 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bab901-28a0-418d-aa14-55851e260156 admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:42:49.724882 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bab901-28a0-418d-aa14-55851e260156 admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:42:49.729782 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba828ca5-1813-45a1-aeef-093509f6c724 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:42:49.730382 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba828ca5-1813-45a1-aeef-093509f6c724 admin admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:49.730654 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba828ca5-1813-45a1-aeef-093509f6c724 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:49.730868 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba828ca5-1813-45a1-aeef-093509f6c724 admin admin] PATH_INFO: `/v3/roles` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:49.731565 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba828ca5-1813-45a1-aeef-093509f6c724 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:42:49.734874 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba828ca5-1813-45a1-aeef-093509f6c724 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:42:49.747916 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1150/2286] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:42:49 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:42:49.749283 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b08edaa-8573-4179-9e45-e85d961d775c admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:42:49.759500 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b08edaa-8573-4179-9e45-e85d961d775c admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:42:49.764144 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab60d2f-0904-4bc9-b65e-78bd0b05cace admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:42:49.764872 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab60d2f-0904-4bc9-b65e-78bd0b05cace admin admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:49.765679 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab60d2f-0904-4bc9-b65e-78bd0b05cace admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:49.765976 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab60d2f-0904-4bc9-b65e-78bd0b05cace admin admin] PATH_INFO: `/v3/roles` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:49.766886 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab60d2f-0904-4bc9-b65e-78bd0b05cace admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:42:49.769310 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab60d2f-0904-4bc9-b65e-78bd0b05cace admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:42:49.783708 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1137/2287] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:42:49 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:42:49.785820 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba828ca5-1813-45a1-aeef-093509f6c724 admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:42:49.796001 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba828ca5-1813-45a1-aeef-093509f6c724 admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:42:49.801548 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f11ace-0790-4d4e-92a1-4efee16fc964 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:42:49.802212 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f11ace-0790-4d4e-92a1-4efee16fc964 admin admin] REQUEST_METHOD: `PUT` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:49.802567 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f11ace-0790-4d4e-92a1-4efee16fc964 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:49.802765 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f11ace-0790-4d4e-92a1-4efee16fc964 admin admin] PATH_INFO: `/v3/projects/05609434a81e4cb5a192125eb25ffc30/users/245b8d59902c4f438abc5fd9e38eb1bf/roles/02476a5b82eb4b188c18a5d706a3ef68` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:49.818084 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f11ace-0790-4d4e-92a1-4efee16fc964 admin admin] RBAC: Authorizing `identity:create_grant(project_id=05609434a81e4cb5a192125eb25ffc30, user_id=245b8d59902c4f438abc5fd9e38eb1bf, role_id=02476a5b82eb4b188c18a5d706a3ef68)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:42:49.820784 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f11ace-0790-4d4e-92a1-4efee16fc964 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:42:49.835199 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1151/2288] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 17:42:49 2022] PUT /identity/v3/projects/05609434a81e4cb5a192125eb25ffc30/users/245b8d59902c4f438abc5fd9e38eb1bf/roles/02476a5b82eb4b188c18a5d706a3ef68 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:42:49.847249 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017988d3-cf5c-40cc-ad8f-67649bb7ae71 None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:49.847537 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017988d3-cf5c-40cc-ad8f-67649bb7ae71 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:49.847817 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017988d3-cf5c-40cc-ad8f-67649bb7ae71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:49.876348 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-017988d3-cf5c-40cc-ad8f-67649bb7ae71 None None] MFA Rules not processed for user `245b8d59902c4f438abc5fd9e38eb1bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:42:49.877704 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-017988d3-cf5c-40cc-ad8f-67649bb7ae71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:49.906417 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:42:49.953611 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1138/2289] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:42:49 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:42:49.958478 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f11ace-0790-4d4e-92a1-4efee16fc964 admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:42:50.032582 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f11ace-0790-4d4e-92a1-4efee16fc964 admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:42:50.037451 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00b86b6-31b7-4405-af4b-deeae5a02249 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:42:50.038051 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00b86b6-31b7-4405-af4b-deeae5a02249 admin admin] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:50.038329 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00b86b6-31b7-4405-af4b-deeae5a02249 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:50.038602 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00b86b6-31b7-4405-af4b-deeae5a02249 admin admin] PATH_INFO: `/v3/projects` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:50.039244 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00b86b6-31b7-4405-af4b-deeae5a02249 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:42:50.041182 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00b86b6-31b7-4405-af4b-deeae5a02249 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:42:50.070138 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1152/2290] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 17:42:49 2022] POST /identity/v3/projects => generated 377 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:42:50.074125 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017988d3-cf5c-40cc-ad8f-67649bb7ae71 None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:42:50.085524 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017988d3-cf5c-40cc-ad8f-67649bb7ae71 None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:42:50.149336 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e1b011-d10f-4918-9035-e26f7a1cd09e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:42:50.149943 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e1b011-d10f-4918-9035-e26f7a1cd09e admin admin] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:50.150221 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e1b011-d10f-4918-9035-e26f7a1cd09e admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:50.150491 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e1b011-d10f-4918-9035-e26f7a1cd09e admin admin] PATH_INFO: `/v3/users` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:50.151114 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e1b011-d10f-4918-9035-e26f7a1cd09e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:42:50.153541 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e1b011-d10f-4918-9035-e26f7a1cd09e admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:42:50.178588 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1139/2291] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 17:42:50 2022] POST /identity/v3/users => generated 295 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:42:50.185612 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00b86b6-31b7-4405-af4b-deeae5a02249 admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:42:50.198832 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00b86b6-31b7-4405-af4b-deeae5a02249 admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:42:50.204194 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444d4836-2de1-427e-b68d-0ebc4ed6d446 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:42:50.204947 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444d4836-2de1-427e-b68d-0ebc4ed6d446 admin admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:50.205231 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444d4836-2de1-427e-b68d-0ebc4ed6d446 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:50.205533 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444d4836-2de1-427e-b68d-0ebc4ed6d446 admin admin] PATH_INFO: `/v3/roles` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:50.206176 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444d4836-2de1-427e-b68d-0ebc4ed6d446 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:42:50.207814 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444d4836-2de1-427e-b68d-0ebc4ed6d446 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:42:50.219270 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1153/2292] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:42:50 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:42:50.223458 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e1b011-d10f-4918-9035-e26f7a1cd09e admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:42:50.233555 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e1b011-d10f-4918-9035-e26f7a1cd09e admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:42:50.238216 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbaca77-79a9-4aa0-8583-977e0fa85f6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:42:50.238861 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbaca77-79a9-4aa0-8583-977e0fa85f6b admin admin] REQUEST_METHOD: `PUT` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:50.239140 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbaca77-79a9-4aa0-8583-977e0fa85f6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:50.239414 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbaca77-79a9-4aa0-8583-977e0fa85f6b admin admin] PATH_INFO: `/v3/projects/12f6c02044f34cafa2d5e946a14f0907/users/459116c86b304d07ae9f96918594b1f3/roles/e411ad9eb96e4e8d85b59b5b700ada7b` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:50.254870 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbaca77-79a9-4aa0-8583-977e0fa85f6b admin admin] RBAC: Authorizing `identity:create_grant(project_id=12f6c02044f34cafa2d5e946a14f0907, user_id=459116c86b304d07ae9f96918594b1f3, role_id=e411ad9eb96e4e8d85b59b5b700ada7b)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:42:50.257588 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbaca77-79a9-4aa0-8583-977e0fa85f6b admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:42:50.270692 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1140/2293] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 17:42:50 2022] PUT /identity/v3/projects/12f6c02044f34cafa2d5e946a14f0907/users/459116c86b304d07ae9f96918594b1f3/roles/e411ad9eb96e4e8d85b59b5b700ada7b => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:42:50.274784 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444d4836-2de1-427e-b68d-0ebc4ed6d446 admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:42:50.284542 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444d4836-2de1-427e-b68d-0ebc4ed6d446 admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:42:50.342757 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191ad4ae-330c-4c71-9108-097341251b6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:42:50.343426 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191ad4ae-330c-4c71-9108-097341251b6a admin admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:50.343708 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191ad4ae-330c-4c71-9108-097341251b6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:50.343986 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191ad4ae-330c-4c71-9108-097341251b6a admin admin] PATH_INFO: `/v3/roles` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:50.344698 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191ad4ae-330c-4c71-9108-097341251b6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:42:50.346329 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191ad4ae-330c-4c71-9108-097341251b6a admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:42:50.362567 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1154/2294] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:42:50 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:42:50.364035 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbaca77-79a9-4aa0-8583-977e0fa85f6b admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:42:50.375723 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbaca77-79a9-4aa0-8583-977e0fa85f6b admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:42:50.380753 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdfaf43-a470-4b77-a5c7-0e2a15efa1ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:42:50.381354 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdfaf43-a470-4b77-a5c7-0e2a15efa1ee admin admin] REQUEST_METHOD: `PUT` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:50.381633 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdfaf43-a470-4b77-a5c7-0e2a15efa1ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:50.381903 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdfaf43-a470-4b77-a5c7-0e2a15efa1ee admin admin] PATH_INFO: `/v3/projects/12f6c02044f34cafa2d5e946a14f0907/users/459116c86b304d07ae9f96918594b1f3/roles/e411ad9eb96e4e8d85b59b5b700ada7b` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:50.383838 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdfaf43-a470-4b77-a5c7-0e2a15efa1ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=12f6c02044f34cafa2d5e946a14f0907, user_id=459116c86b304d07ae9f96918594b1f3, role_id=e411ad9eb96e4e8d85b59b5b700ada7b)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:42:50.386554 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdfaf43-a470-4b77-a5c7-0e2a15efa1ee admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:42:50.398105 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1141/2295] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 17:42:50 2022] PUT /identity/v3/projects/12f6c02044f34cafa2d5e946a14f0907/users/459116c86b304d07ae9f96918594b1f3/roles/e411ad9eb96e4e8d85b59b5b700ada7b => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:42:50.409194 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36158ac1-1200-421f-9d66-c7d1d8c41c9a None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:50.409510 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36158ac1-1200-421f-9d66-c7d1d8c41c9a None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:50.409784 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36158ac1-1200-421f-9d66-c7d1d8c41c9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:50.435707 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:42:50.442729 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-36158ac1-1200-421f-9d66-c7d1d8c41c9a None None] MFA Rules not processed for user `459116c86b304d07ae9f96918594b1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:42:50.444803 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-36158ac1-1200-421f-9d66-c7d1d8c41c9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:50.548709 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1155/2296] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:42:50 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:42:50.550143 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdfaf43-a470-4b77-a5c7-0e2a15efa1ee admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:42:50.552260 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-9cdfaf43-a470-4b77-a5c7-0e2a15efa1ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:50.556451 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36158ac1-1200-421f-9d66-c7d1d8c41c9a None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:42:50.558075 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-36158ac1-1200-421f-9d66-c7d1d8c41c9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:50.648647 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdfaf43-a470-4b77-a5c7-0e2a15efa1ee admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:42:50.648894 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36158ac1-1200-421f-9d66-c7d1d8c41c9a None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:42:50.654021 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050dc2e7-01ac-4c06-9b98-b0801f50d799 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:42:50.654680 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050dc2e7-01ac-4c06-9b98-b0801f50d799 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:50.654958 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050dc2e7-01ac-4c06-9b98-b0801f50d799 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:50.655232 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050dc2e7-01ac-4c06-9b98-b0801f50d799 admin admin] PATH_INFO: `/v3/users/ccb4c2a7b74a479d890b73a0aae33bad` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:50.655434 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f08d84-d6db-4926-8d41-f8c062328216 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59be342541264942ad87f60ff4de6c8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:42:50.656078 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f08d84-d6db-4926-8d41-f8c062328216 service cinder] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:50.656475 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f08d84-d6db-4926-8d41-f8c062328216 service cinder] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:50.656627 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050dc2e7-01ac-4c06-9b98-b0801f50d799 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ccb4c2a7b74a479d890b73a0aae33bad)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:42:50.656720 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f08d84-d6db-4926-8d41-f8c062328216 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:50.658419 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050dc2e7-01ac-4c06-9b98-b0801f50d799 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:42:50.685274 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:42:50.724405 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-050dc2e7-01ac-4c06-9b98-b0801f50d799 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ccb4c2a7b74a479d890b73a0aae33bad'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:42:50.724708 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f08d84-d6db-4926-8d41-f8c062328216 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:42:50.725707 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-050dc2e7-01ac-4c06-9b98-b0801f50d799 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ccb4c2a7b74a479d890b73a0aae33bad'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:42:50.726265 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f08d84-d6db-4926-8d41-f8c062328216 service cinder] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:42:50.735086 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-050dc2e7-01ac-4c06-9b98-b0801f50d799 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ccb4c2a7b74a479d890b73a0aae33bad'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:42:50.735792 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-050dc2e7-01ac-4c06-9b98-b0801f50d799 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ccb4c2a7b74a479d890b73a0aae33bad'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:42:50.737705 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1156/2297] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:42:50 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 17:42:50.745227 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-050dc2e7-01ac-4c06-9b98-b0801f50d799 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ccb4c2a7b74a479d890b73a0aae33bad'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:42:50.771440 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1142/2298] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 17:42:50 2022] DELETE /identity/v3/users/ccb4c2a7b74a479d890b73a0aae33bad => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:42:50.777954 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2af3e4e-0679-4fc7-bcee-de44624d01d9 None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:50.778236 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2af3e4e-0679-4fc7-bcee-de44624d01d9 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:50.778515 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2af3e4e-0679-4fc7-bcee-de44624d01d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:50.815903 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-f2af3e4e-0679-4fc7-bcee-de44624d01d9 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:42:50.816447 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-f2af3e4e-0679-4fc7-bcee-de44624d01d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:50.879971 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:42:50.893677 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1157/2299] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:42:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:42:50.904676 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050dc2e7-01ac-4c06-9b98-b0801f50d799 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:42:50.907037 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-050dc2e7-01ac-4c06-9b98-b0801f50d799 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:50.973819 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050dc2e7-01ac-4c06-9b98-b0801f50d799 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:42:50.977790 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18668043-91eb-4104-b073-af3c2e8f08bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43ad98a250a144099ccd18378aedd4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:42:50.978407 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18668043-91eb-4104-b073-af3c2e8f08bf service neutron] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:50.978681 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18668043-91eb-4104-b073-af3c2e8f08bf service neutron] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:50.979106 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18668043-91eb-4104-b073-af3c2e8f08bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:50.989447 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18668043-91eb-4104-b073-af3c2e8f08bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:42:50.993808 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18668043-91eb-4104-b073-af3c2e8f08bf service neutron] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:42:51.002402 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1143/2300] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:42:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:42:51.194656 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf61273d-8186-49c5-a67e-7266db25a97e None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:51.195147 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf61273d-8186-49c5-a67e-7266db25a97e None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:51.195238 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf61273d-8186-49c5-a67e-7266db25a97e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:51.214665 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-cf61273d-8186-49c5-a67e-7266db25a97e None None] MFA Rules not processed for user `757f4cb8bb714ccc90b2f2589f9b2a1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:42:51.215964 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-cf61273d-8186-49c5-a67e-7266db25a97e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:51.244060 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18668043-91eb-4104-b073-af3c2e8f08bf service neutron] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:42:51.246052 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-18668043-91eb-4104-b073-af3c2e8f08bf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:51.270506 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18668043-91eb-4104-b073-af3c2e8f08bf service neutron] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:42:51.275360 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fffa40f-affb-4c16-b46d-021db2bf79b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:42:51.275991 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fffa40f-affb-4c16-b46d-021db2bf79b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:51.276264 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fffa40f-affb-4c16-b46d-021db2bf79b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:51.276542 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fffa40f-affb-4c16-b46d-021db2bf79b5 admin admin] PATH_INFO: `/v3/projects/376329ca784b4fdea3205bf08edb45a8` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:51.277703 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fffa40f-affb-4c16-b46d-021db2bf79b5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=376329ca784b4fdea3205bf08edb45a8)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:42:51.279614 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fffa40f-affb-4c16-b46d-021db2bf79b5 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:42:51.283648 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1158/2301] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:42:51 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:42:51.328433 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-0fffa40f-affb-4c16-b46d-021db2bf79b5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '376329ca784b4fdea3205bf08edb45a8', 'request_id': 'req-0fffa40f-affb-4c16-b46d-021db2bf79b5'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:42:51.334571 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-0fffa40f-affb-4c16-b46d-021db2bf79b5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '376329ca784b4fdea3205bf08edb45a8', 'request_id': 'req-0fffa40f-affb-4c16-b46d-021db2bf79b5'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:42:51.342943 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-0fffa40f-affb-4c16-b46d-021db2bf79b5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '376329ca784b4fdea3205bf08edb45a8', 'request_id': 'req-0fffa40f-affb-4c16-b46d-021db2bf79b5'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:42:51.355715 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-0fffa40f-affb-4c16-b46d-021db2bf79b5 admin admin] The token cache is being invalidate because project 376329ca784b4fdea3205bf08edb45a8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81759) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:42:51.356012 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-0fffa40f-affb-4c16-b46d-021db2bf79b5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:42:51.356650 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-0fffa40f-affb-4c16-b46d-021db2bf79b5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:42:51.358203 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1144/2302] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 17:42:51 2022] DELETE /identity/v3/projects/376329ca784b4fdea3205bf08edb45a8 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:42:52.662294 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf61273d-8186-49c5-a67e-7266db25a97e None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:42:52.663787 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-cf61273d-8186-49c5-a67e-7266db25a97e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:52.676233 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf61273d-8186-49c5-a67e-7266db25a97e None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:42:52.738329 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3606ac-e43f-49e7-991d-8b46bb7cba1f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43ad98a250a144099ccd18378aedd4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:42:52.738864 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3606ac-e43f-49e7-991d-8b46bb7cba1f service neutron] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:52.739144 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3606ac-e43f-49e7-991d-8b46bb7cba1f service neutron] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:52.739416 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3606ac-e43f-49e7-991d-8b46bb7cba1f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:52.741137 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-1d3606ac-e43f-49e7-991d-8b46bb7cba1f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:52.825088 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3606ac-e43f-49e7-991d-8b46bb7cba1f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:42:52.826860 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3606ac-e43f-49e7-991d-8b46bb7cba1f service neutron] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:42:52.842573 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1159/2303] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:42:52 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:42:53.433604 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fffa40f-affb-4c16-b46d-021db2bf79b5 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:42:53.435239 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-0fffa40f-affb-4c16-b46d-021db2bf79b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:53.475270 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:42:53.490735 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021a1f8d-91c8-4543-bddf-586f5a30c749 None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:53.491026 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021a1f8d-91c8-4543-bddf-586f5a30c749 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:53.491689 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021a1f8d-91c8-4543-bddf-586f5a30c749 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:53.513307 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-021a1f8d-91c8-4543-bddf-586f5a30c749 None None] MFA Rules not processed for user `40c0367417434c04948e9f2a03e880a9`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:42:53.514629 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-021a1f8d-91c8-4543-bddf-586f5a30c749 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:53.520031 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fffa40f-affb-4c16-b46d-021db2bf79b5 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:42:53.524706 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb017c14-faa8-4f90-a8e8-b58a88d249cc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59be342541264942ad87f60ff4de6c8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:42:53.525231 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb017c14-faa8-4f90-a8e8-b58a88d249cc service cinder] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:53.525516 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb017c14-faa8-4f90-a8e8-b58a88d249cc service cinder] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:53.525792 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb017c14-faa8-4f90-a8e8-b58a88d249cc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:53.527740 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-fb017c14-faa8-4f90-a8e8-b58a88d249cc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:53.567922 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:42:53.582853 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1160/2304] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:42:53 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:42:53.610456 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb017c14-faa8-4f90-a8e8-b58a88d249cc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:42:53.612022 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb017c14-faa8-4f90-a8e8-b58a88d249cc service cinder] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:42:53.630635 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1145/2305] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:42:53 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:42:54.295126 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021a1f8d-91c8-4543-bddf-586f5a30c749 None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:42:54.312488 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021a1f8d-91c8-4543-bddf-586f5a30c749 None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:42:54.316219 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b3812e-1ba8-454f-b53c-951969952e16 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59be342541264942ad87f60ff4de6c8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:42:54.316860 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b3812e-1ba8-454f-b53c-951969952e16 service cinder] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:54.317147 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b3812e-1ba8-454f-b53c-951969952e16 service cinder] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:54.317424 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b3812e-1ba8-454f-b53c-951969952e16 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:54.319173 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-d9b3812e-1ba8-454f-b53c-951969952e16 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:54.390802 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b3812e-1ba8-454f-b53c-951969952e16 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:42:54.392521 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b3812e-1ba8-454f-b53c-951969952e16 service cinder] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:42:54.402652 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1161/2306] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:42:54 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:42:59.406360 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb017c14-faa8-4f90-a8e8-b58a88d249cc service cinder] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:42:59.407909 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-fb017c14-faa8-4f90-a8e8-b58a88d249cc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:59.473722 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb017c14-faa8-4f90-a8e8-b58a88d249cc service cinder] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:42:59.478196 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efddea9-1f79-485d-bf62-1a8144ac8720 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:42:59.478814 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efddea9-1f79-485d-bf62-1a8144ac8720 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:59.479092 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efddea9-1f79-485d-bf62-1a8144ac8720 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:59.479366 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efddea9-1f79-485d-bf62-1a8144ac8720 admin admin] PATH_INFO: `/v3/users/6e6ef8b7c265474f888d7b78fb9e2bb7` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:59.480477 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efddea9-1f79-485d-bf62-1a8144ac8720 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e6ef8b7c265474f888d7b78fb9e2bb7)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:42:59.482420 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efddea9-1f79-485d-bf62-1a8144ac8720 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:42:59.527621 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-4efddea9-1f79-485d-bf62-1a8144ac8720 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e6ef8b7c265474f888d7b78fb9e2bb7'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:42:59.528274 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-4efddea9-1f79-485d-bf62-1a8144ac8720 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e6ef8b7c265474f888d7b78fb9e2bb7'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:42:59.535881 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-4efddea9-1f79-485d-bf62-1a8144ac8720 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e6ef8b7c265474f888d7b78fb9e2bb7'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:42:59.536445 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-4efddea9-1f79-485d-bf62-1a8144ac8720 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e6ef8b7c265474f888d7b78fb9e2bb7'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:42:59.543575 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-4efddea9-1f79-485d-bf62-1a8144ac8720 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e6ef8b7c265474f888d7b78fb9e2bb7'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:42:59.556082 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:42:59.561513 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1146/2307] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 17:42:59 2022] DELETE /identity/v3/users/6e6ef8b7c265474f888d7b78fb9e2bb7 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:42:59.566093 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75b96da-1748-48cf-89e3-52c848f5be0d None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:59.566580 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75b96da-1748-48cf-89e3-52c848f5be0d None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:59.566880 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75b96da-1748-48cf-89e3-52c848f5be0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:59.586174 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-a75b96da-1748-48cf-89e3-52c848f5be0d None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:42:59.587360 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-a75b96da-1748-48cf-89e3-52c848f5be0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:59.644177 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1162/2308] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:42:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:42:59.653738 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efddea9-1f79-485d-bf62-1a8144ac8720 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:42:59.655120 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-4efddea9-1f79-485d-bf62-1a8144ac8720 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:59.711615 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efddea9-1f79-485d-bf62-1a8144ac8720 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:42:59.715369 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f107c5-92c6-46ce-b797-d4c8e42acae6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43ad98a250a144099ccd18378aedd4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:42:59.715971 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f107c5-92c6-46ce-b797-d4c8e42acae6 service neutron] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:59.716278 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f107c5-92c6-46ce-b797-d4c8e42acae6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:59.716565 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f107c5-92c6-46ce-b797-d4c8e42acae6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:59.725507 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f107c5-92c6-46ce-b797-d4c8e42acae6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:42:59.726927 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f107c5-92c6-46ce-b797-d4c8e42acae6 service neutron] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:42:59.736731 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1147/2309] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:42:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:43:00.083730 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75b96da-1748-48cf-89e3-52c848f5be0d None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:43:00.085209 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-a75b96da-1748-48cf-89e3-52c848f5be0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:43:00.097711 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75b96da-1748-48cf-89e3-52c848f5be0d None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:43:00.102389 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5cdec9-4435-4af3-b6d3-233a76ee4c78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:43:00.103024 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5cdec9-4435-4af3-b6d3-233a76ee4c78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:43:00.103306 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5cdec9-4435-4af3-b6d3-233a76ee4c78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:43:00.103582 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5cdec9-4435-4af3-b6d3-233a76ee4c78 admin admin] PATH_INFO: `/v3/projects/b0643edf95e84451b9ffd8fc79891803` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:43:00.104742 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5cdec9-4435-4af3-b6d3-233a76ee4c78 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b0643edf95e84451b9ffd8fc79891803)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:43:00.106658 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5cdec9-4435-4af3-b6d3-233a76ee4c78 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:43:00.152652 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-9a5cdec9-4435-4af3-b6d3-233a76ee4c78 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b0643edf95e84451b9ffd8fc79891803', 'request_id': 'req-9a5cdec9-4435-4af3-b6d3-233a76ee4c78'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:43:00.160639 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-9a5cdec9-4435-4af3-b6d3-233a76ee4c78 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b0643edf95e84451b9ffd8fc79891803', 'request_id': 'req-9a5cdec9-4435-4af3-b6d3-233a76ee4c78'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:43:00.166807 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-9a5cdec9-4435-4af3-b6d3-233a76ee4c78 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b0643edf95e84451b9ffd8fc79891803', 'request_id': 'req-9a5cdec9-4435-4af3-b6d3-233a76ee4c78'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:43:00.177893 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-9a5cdec9-4435-4af3-b6d3-233a76ee4c78 admin admin] The token cache is being invalidate because project b0643edf95e84451b9ffd8fc79891803 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81758) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:43:00.178183 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-9a5cdec9-4435-4af3-b6d3-233a76ee4c78 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:43:00.178834 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-9a5cdec9-4435-4af3-b6d3-233a76ee4c78 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:43:00.180306 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1163/2310] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 17:43:00 2022] DELETE /identity/v3/projects/b0643edf95e84451b9ffd8fc79891803 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:43:00.184278 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f107c5-92c6-46ce-b797-d4c8e42acae6 service neutron] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:43:00.193863 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f107c5-92c6-46ce-b797-d4c8e42acae6 service neutron] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:43:00.248473 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31eb9592-3b59-4c8b-8f8b-8fdf1b4e1f54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:43:00.249129 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31eb9592-3b59-4c8b-8f8b-8fdf1b4e1f54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:43:00.249406 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31eb9592-3b59-4c8b-8f8b-8fdf1b4e1f54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:43:00.249675 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31eb9592-3b59-4c8b-8f8b-8fdf1b4e1f54 admin admin] PATH_INFO: `/v3/users/245b8d59902c4f438abc5fd9e38eb1bf` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:43:00.250774 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31eb9592-3b59-4c8b-8f8b-8fdf1b4e1f54 admin admin] RBAC: Authorizing `identity:delete_user(user_id=245b8d59902c4f438abc5fd9e38eb1bf)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:43:00.252786 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31eb9592-3b59-4c8b-8f8b-8fdf1b4e1f54 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:43:00.298737 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-31eb9592-3b59-4c8b-8f8b-8fdf1b4e1f54 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '245b8d59902c4f438abc5fd9e38eb1bf'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:43:00.299404 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-31eb9592-3b59-4c8b-8f8b-8fdf1b4e1f54 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '245b8d59902c4f438abc5fd9e38eb1bf'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:43:00.306146 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-31eb9592-3b59-4c8b-8f8b-8fdf1b4e1f54 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '245b8d59902c4f438abc5fd9e38eb1bf'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:43:00.306770 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-31eb9592-3b59-4c8b-8f8b-8fdf1b4e1f54 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '245b8d59902c4f438abc5fd9e38eb1bf'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:43:00.313595 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-31eb9592-3b59-4c8b-8f8b-8fdf1b4e1f54 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '245b8d59902c4f438abc5fd9e38eb1bf'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:43:00.330804 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1148/2311] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 17:43:00 2022] DELETE /identity/v3/users/245b8d59902c4f438abc5fd9e38eb1bf => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:43:00.608049 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5cdec9-4435-4af3-b6d3-233a76ee4c78 admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:43:00.610161 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-9a5cdec9-4435-4af3-b6d3-233a76ee4c78 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:43:00.626283 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5cdec9-4435-4af3-b6d3-233a76ee4c78 admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:43:00.635305 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:43:00.685958 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44149f50-5da0-4bba-8cb7-9d88ed471494 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:43:00.686613 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44149f50-5da0-4bba-8cb7-9d88ed471494 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:43:00.687411 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44149f50-5da0-4bba-8cb7-9d88ed471494 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:43:00.687682 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44149f50-5da0-4bba-8cb7-9d88ed471494 admin admin] PATH_INFO: `/v3/projects/05609434a81e4cb5a192125eb25ffc30` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:43:00.688849 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44149f50-5da0-4bba-8cb7-9d88ed471494 admin admin] RBAC: Authorizing `identity:delete_project(project_id=05609434a81e4cb5a192125eb25ffc30)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:43:00.690787 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44149f50-5da0-4bba-8cb7-9d88ed471494 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:43:00.737002 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-44149f50-5da0-4bba-8cb7-9d88ed471494 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '05609434a81e4cb5a192125eb25ffc30', 'request_id': 'req-44149f50-5da0-4bba-8cb7-9d88ed471494'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:43:00.744603 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-44149f50-5da0-4bba-8cb7-9d88ed471494 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '05609434a81e4cb5a192125eb25ffc30', 'request_id': 'req-44149f50-5da0-4bba-8cb7-9d88ed471494'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:43:00.751362 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-44149f50-5da0-4bba-8cb7-9d88ed471494 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '05609434a81e4cb5a192125eb25ffc30', 'request_id': 'req-44149f50-5da0-4bba-8cb7-9d88ed471494'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:43:00.762176 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-44149f50-5da0-4bba-8cb7-9d88ed471494 admin admin] The token cache is being invalidate because project 05609434a81e4cb5a192125eb25ffc30 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81758) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:43:00.762463 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-44149f50-5da0-4bba-8cb7-9d88ed471494 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:43:00.763101 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-44149f50-5da0-4bba-8cb7-9d88ed471494 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:43:00.764666 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1164/2312] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 17:43:00 2022] DELETE /identity/v3/projects/05609434a81e4cb5a192125eb25ffc30 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:43:00.769502 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31eb9592-3b59-4c8b-8f8b-8fdf1b4e1f54 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:43:00.771441 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-31eb9592-3b59-4c8b-8f8b-8fdf1b4e1f54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:43:00.784676 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31eb9592-3b59-4c8b-8f8b-8fdf1b4e1f54 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:43:00.801863 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:43:00.843769 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab92bd86-a2e2-432e-98b6-fdbaf38708e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:43:00.844339 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab92bd86-a2e2-432e-98b6-fdbaf38708e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:43:00.844639 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab92bd86-a2e2-432e-98b6-fdbaf38708e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:43:00.844913 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab92bd86-a2e2-432e-98b6-fdbaf38708e3 admin admin] PATH_INFO: `/v3/users/459116c86b304d07ae9f96918594b1f3` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:43:00.846039 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab92bd86-a2e2-432e-98b6-fdbaf38708e3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=459116c86b304d07ae9f96918594b1f3)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:43:00.847964 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab92bd86-a2e2-432e-98b6-fdbaf38708e3 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:43:00.894617 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-ab92bd86-a2e2-432e-98b6-fdbaf38708e3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '459116c86b304d07ae9f96918594b1f3'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:43:00.895254 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-ab92bd86-a2e2-432e-98b6-fdbaf38708e3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '459116c86b304d07ae9f96918594b1f3'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:43:00.902151 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-ab92bd86-a2e2-432e-98b6-fdbaf38708e3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '459116c86b304d07ae9f96918594b1f3'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:43:00.902806 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-ab92bd86-a2e2-432e-98b6-fdbaf38708e3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '459116c86b304d07ae9f96918594b1f3'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:43:00.911610 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-ab92bd86-a2e2-432e-98b6-fdbaf38708e3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '459116c86b304d07ae9f96918594b1f3'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:43:00.928847 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1149/2313] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 17:43:00 2022] DELETE /identity/v3/users/459116c86b304d07ae9f96918594b1f3 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:43:01.240085 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44149f50-5da0-4bba-8cb7-9d88ed471494 admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:43:01.241670 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-44149f50-5da0-4bba-8cb7-9d88ed471494 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:43:01.254402 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44149f50-5da0-4bba-8cb7-9d88ed471494 admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:43:01.312309 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975a126d-ffc0-4086-9648-7d915c72270a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:43:01.313112 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975a126d-ffc0-4086-9648-7d915c72270a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:43:01.313392 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975a126d-ffc0-4086-9648-7d915c72270a admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:43:01.313666 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975a126d-ffc0-4086-9648-7d915c72270a admin admin] PATH_INFO: `/v3/projects/12f6c02044f34cafa2d5e946a14f0907` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:43:01.314806 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975a126d-ffc0-4086-9648-7d915c72270a admin admin] RBAC: Authorizing `identity:delete_project(project_id=12f6c02044f34cafa2d5e946a14f0907)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:43:01.317198 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975a126d-ffc0-4086-9648-7d915c72270a admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:43:01.362403 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:43:01.366328 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-975a126d-ffc0-4086-9648-7d915c72270a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '12f6c02044f34cafa2d5e946a14f0907', 'request_id': 'req-975a126d-ffc0-4086-9648-7d915c72270a'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:43:01.375441 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-975a126d-ffc0-4086-9648-7d915c72270a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '12f6c02044f34cafa2d5e946a14f0907', 'request_id': 'req-975a126d-ffc0-4086-9648-7d915c72270a'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:43:01.382020 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-975a126d-ffc0-4086-9648-7d915c72270a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '12f6c02044f34cafa2d5e946a14f0907', 'request_id': 'req-975a126d-ffc0-4086-9648-7d915c72270a'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:43:01.393806 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-975a126d-ffc0-4086-9648-7d915c72270a admin admin] The token cache is being invalidate because project 12f6c02044f34cafa2d5e946a14f0907 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81758) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:43:01.394095 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-975a126d-ffc0-4086-9648-7d915c72270a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:43:01.394766 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-975a126d-ffc0-4086-9648-7d915c72270a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:43:01.399565 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1165/2314] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 17:43:01 2022] DELETE /identity/v3/projects/12f6c02044f34cafa2d5e946a14f0907 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:43:01.401342 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98eb9607-ce73-40a1-8f44-a01d901b55a5 None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:43:01.401630 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98eb9607-ce73-40a1-8f44-a01d901b55a5 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:43:01.401907 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98eb9607-ce73-40a1-8f44-a01d901b55a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:43:01.421940 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-98eb9607-ce73-40a1-8f44-a01d901b55a5 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:43:01.423250 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-98eb9607-ce73-40a1-8f44-a01d901b55a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:43:01.486452 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1150/2315] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:43:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:43:01.495514 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae96219-a635-4798-b368-d1f99b89f1c0 None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:43:01.495879 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae96219-a635-4798-b368-d1f99b89f1c0 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:43:01.496162 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae96219-a635-4798-b368-d1f99b89f1c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:43:01.517406 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-5ae96219-a635-4798-b368-d1f99b89f1c0 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:43:01.518705 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-5ae96219-a635-4798-b368-d1f99b89f1c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:43:01.527075 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1166/2316] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:43:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:43:01.531505 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98eb9607-ce73-40a1-8f44-a01d901b55a5 None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:43:01.541140 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98eb9607-ce73-40a1-8f44-a01d901b55a5 None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:43:01.545842 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695082a2-0970-4608-9fdd-eaffe07697ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:43:01.546478 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695082a2-0970-4608-9fdd-eaffe07697ae admin admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:43:01.546758 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695082a2-0970-4608-9fdd-eaffe07697ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:43:01.547039 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695082a2-0970-4608-9fdd-eaffe07697ae admin admin] PATH_INFO: `/v3/domains` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:43:01.547686 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695082a2-0970-4608-9fdd-eaffe07697ae admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:43:01.547990 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695082a2-0970-4608-9fdd-eaffe07697ae admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:43:01.549627 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695082a2-0970-4608-9fdd-eaffe07697ae admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:43:01.561249 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1151/2317] 10.222.0.45 () {58 vars in 1153 bytes} [Wed Aug 24 17:43:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:43:01.566191 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae96219-a635-4798-b368-d1f99b89f1c0 None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:43:01.575921 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae96219-a635-4798-b368-d1f99b89f1c0 None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:43:01.581949 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e0d3a5-176d-4079-a1e7-28154bfcfc8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:43:01.582564 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e0d3a5-176d-4079-a1e7-28154bfcfc8f admin admin] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:43:01.582847 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e0d3a5-176d-4079-a1e7-28154bfcfc8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:43:01.583125 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e0d3a5-176d-4079-a1e7-28154bfcfc8f admin admin] PATH_INFO: `/v3/projects` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:43:01.583843 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e0d3a5-176d-4079-a1e7-28154bfcfc8f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:43:01.586056 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e0d3a5-176d-4079-a1e7-28154bfcfc8f admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:43:01.616984 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1167/2318] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 17:43:01 2022] POST /identity/v3/projects => generated 391 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:43:01.618964 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695082a2-0970-4608-9fdd-eaffe07697ae admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:43:01.630198 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695082a2-0970-4608-9fdd-eaffe07697ae admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:43:01.656788 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:43:01.695024 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a917338e-97a6-4b93-a9e7-e0d42ab8ce47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:43:01.695657 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a917338e-97a6-4b93-a9e7-e0d42ab8ce47 admin admin] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:43:01.695967 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a917338e-97a6-4b93-a9e7-e0d42ab8ce47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:43:01.696237 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a917338e-97a6-4b93-a9e7-e0d42ab8ce47 admin admin] PATH_INFO: `/v3/users` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:43:01.697066 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a917338e-97a6-4b93-a9e7-e0d42ab8ce47 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:43:01.698905 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a917338e-97a6-4b93-a9e7-e0d42ab8ce47 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:43:01.721270 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1152/2319] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 17:43:01 2022] POST /identity/v3/users => generated 296 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:43:01.725192 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e0d3a5-176d-4079-a1e7-28154bfcfc8f admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:43:01.736386 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e0d3a5-176d-4079-a1e7-28154bfcfc8f admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:43:01.741785 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e7d078-dd98-46ec-be2d-182a031fed61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:43:01.742105 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e7d078-dd98-46ec-be2d-182a031fed61 admin admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:43:01.742488 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e7d078-dd98-46ec-be2d-182a031fed61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:43:01.742897 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e7d078-dd98-46ec-be2d-182a031fed61 admin admin] PATH_INFO: `/v3/roles` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:43:01.743649 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e7d078-dd98-46ec-be2d-182a031fed61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:43:01.745479 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e7d078-dd98-46ec-be2d-182a031fed61 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:43:01.756661 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1168/2320] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:43:01 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 17:43:01.760803 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a917338e-97a6-4b93-a9e7-e0d42ab8ce47 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:43:01.772536 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a917338e-97a6-4b93-a9e7-e0d42ab8ce47 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:43:01.779451 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0b35cf-8783-4845-bc98-402ae67e224b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:43:01.780363 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0b35cf-8783-4845-bc98-402ae67e224b admin admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:43:01.780805 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0b35cf-8783-4845-bc98-402ae67e224b admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:43:01.781479 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0b35cf-8783-4845-bc98-402ae67e224b admin admin] PATH_INFO: `/v3/roles` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:43:01.782244 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0b35cf-8783-4845-bc98-402ae67e224b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:43:01.783961 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0b35cf-8783-4845-bc98-402ae67e224b admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:43:01.797462 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1153/2321] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:43:01 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:43:01.803377 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e7d078-dd98-46ec-be2d-182a031fed61 admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:43:01.816263 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e7d078-dd98-46ec-be2d-182a031fed61 admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:43:01.830239 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac363ec-4571-43c3-ab50-274cc4c9b7dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:43:01.831758 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac363ec-4571-43c3-ab50-274cc4c9b7dd admin admin] REQUEST_METHOD: `PUT` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:43:01.832407 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac363ec-4571-43c3-ab50-274cc4c9b7dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:43:01.833927 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac363ec-4571-43c3-ab50-274cc4c9b7dd admin admin] PATH_INFO: `/v3/projects/1cd818aebfc649e8b6f46c64f0120c61/users/35d73e24ba584676b9d590ba3ab153a6/roles/02476a5b82eb4b188c18a5d706a3ef68` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:43:01.862741 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac363ec-4571-43c3-ab50-274cc4c9b7dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=1cd818aebfc649e8b6f46c64f0120c61, user_id=35d73e24ba584676b9d590ba3ab153a6, role_id=02476a5b82eb4b188c18a5d706a3ef68)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:43:01.865581 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac363ec-4571-43c3-ab50-274cc4c9b7dd admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:43:01.888646 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1169/2322] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 17:43:01 2022] PUT /identity/v3/projects/1cd818aebfc649e8b6f46c64f0120c61/users/35d73e24ba584676b9d590ba3ab153a6/roles/02476a5b82eb4b188c18a5d706a3ef68 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:43:01.895174 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db26507-aae2-47d5-8183-3ea729f78f05 None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:43:01.895462 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db26507-aae2-47d5-8183-3ea729f78f05 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:43:01.895740 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db26507-aae2-47d5-8183-3ea729f78f05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:43:01.928021 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-9db26507-aae2-47d5-8183-3ea729f78f05 None None] MFA Rules not processed for user `35d73e24ba584676b9d590ba3ab153a6`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:43:01.929469 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-9db26507-aae2-47d5-8183-3ea729f78f05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:43:01.996452 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1154/2323] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:43:01 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:43:02.008306 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac363ec-4571-43c3-ab50-274cc4c9b7dd admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:43:02.010087 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-9ac363ec-4571-43c3-ab50-274cc4c9b7dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:43:02.070813 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac363ec-4571-43c3-ab50-274cc4c9b7dd admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:43:02.075181 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc5b15c-fe4b-45f0-a5d1-97b5c9506749 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59be342541264942ad87f60ff4de6c8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:43:02.076454 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc5b15c-fe4b-45f0-a5d1-97b5c9506749 service cinder] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:43:02.076454 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc5b15c-fe4b-45f0-a5d1-97b5c9506749 service cinder] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:43:02.076642 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc5b15c-fe4b-45f0-a5d1-97b5c9506749 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:43:02.085122 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc5b15c-fe4b-45f0-a5d1-97b5c9506749 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:43:02.086594 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc5b15c-fe4b-45f0-a5d1-97b5c9506749 service cinder] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:43:02.096074 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1170/2324] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:43:02 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:43:11.993446 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=54,keystone:INSERT=3 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:43:12.091321 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=61,keystone:DELETE=4,keystone:INSERT=3 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:43:31.322861 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db26507-aae2-47d5-8183-3ea729f78f05 None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:43:31.324514 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-9db26507-aae2-47d5-8183-3ea729f78f05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:43:31.416815 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db26507-aae2-47d5-8183-3ea729f78f05 None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:43:31.422710 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb75f956-c23e-43ea-a352-4072d7ea9781 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37def4721b554b9ebb7704a19642d15b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:43:31.424169 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb75f956-c23e-43ea-a352-4072d7ea9781 service placement] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:43:31.424571 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb75f956-c23e-43ea-a352-4072d7ea9781 service placement] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:43:31.424930 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb75f956-c23e-43ea-a352-4072d7ea9781 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:43:31.427014 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-bb75f956-c23e-43ea-a352-4072d7ea9781 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:43:31.443294 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb75f956-c23e-43ea-a352-4072d7ea9781 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:43:31.445808 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb75f956-c23e-43ea-a352-4072d7ea9781 service placement] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:43:31.468874 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1155/2325] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:43:31 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:43:33.024230 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc5b15c-fe4b-45f0-a5d1-97b5c9506749 service cinder] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:43:33.109791 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc5b15c-fe4b-45f0-a5d1-97b5c9506749 service cinder] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:43:33.119522 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abec9f41-600b-45b7-8bbc-a94ee7e9b755 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:43:33.120166 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abec9f41-600b-45b7-8bbc-a94ee7e9b755 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:43:33.120447 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abec9f41-600b-45b7-8bbc-a94ee7e9b755 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:43:33.120758 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abec9f41-600b-45b7-8bbc-a94ee7e9b755 admin admin] PATH_INFO: `/v3/users/35d73e24ba584676b9d590ba3ab153a6` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:43:33.121921 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abec9f41-600b-45b7-8bbc-a94ee7e9b755 admin admin] RBAC: Authorizing `identity:delete_user(user_id=35d73e24ba584676b9d590ba3ab153a6)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:43:33.126346 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abec9f41-600b-45b7-8bbc-a94ee7e9b755 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:43:33.186330 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-abec9f41-600b-45b7-8bbc-a94ee7e9b755 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '35d73e24ba584676b9d590ba3ab153a6'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:43:33.187159 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-abec9f41-600b-45b7-8bbc-a94ee7e9b755 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '35d73e24ba584676b9d590ba3ab153a6'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:43:33.197815 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-abec9f41-600b-45b7-8bbc-a94ee7e9b755 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '35d73e24ba584676b9d590ba3ab153a6'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:43:33.199071 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-abec9f41-600b-45b7-8bbc-a94ee7e9b755 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '35d73e24ba584676b9d590ba3ab153a6'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:43:33.228297 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-abec9f41-600b-45b7-8bbc-a94ee7e9b755 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '35d73e24ba584676b9d590ba3ab153a6'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:43:33.257254 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1171/2326] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 17:43:33 2022] DELETE /identity/v3/users/35d73e24ba584676b9d590ba3ab153a6 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:43:33.261725 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd63b5a5-c5ef-4d3e-83a7-3e2d14e7fb24 None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:43:33.262553 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd63b5a5-c5ef-4d3e-83a7-3e2d14e7fb24 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:43:33.263039 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd63b5a5-c5ef-4d3e-83a7-3e2d14e7fb24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:43:33.285117 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-fd63b5a5-c5ef-4d3e-83a7-3e2d14e7fb24 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:43:33.287595 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-fd63b5a5-c5ef-4d3e-83a7-3e2d14e7fb24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:43:33.369029 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1156/2327] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:43:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:43:33.381301 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abec9f41-600b-45b7-8bbc-a94ee7e9b755 admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:43:33.383121 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-abec9f41-600b-45b7-8bbc-a94ee7e9b755 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:43:33.468269 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abec9f41-600b-45b7-8bbc-a94ee7e9b755 admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:43:33.474494 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facf1d3a-b960-4bbf-a482-ccc6c5df9766 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43ad98a250a144099ccd18378aedd4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:43:33.475975 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facf1d3a-b960-4bbf-a482-ccc6c5df9766 service neutron] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:43:33.477076 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facf1d3a-b960-4bbf-a482-ccc6c5df9766 service neutron] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:43:33.477574 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facf1d3a-b960-4bbf-a482-ccc6c5df9766 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:43:33.491863 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facf1d3a-b960-4bbf-a482-ccc6c5df9766 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:43:33.493378 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facf1d3a-b960-4bbf-a482-ccc6c5df9766 service neutron] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:43:33.505876 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1172/2328] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:43:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:43:33.838543 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd63b5a5-c5ef-4d3e-83a7-3e2d14e7fb24 None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:43:33.839975 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-fd63b5a5-c5ef-4d3e-83a7-3e2d14e7fb24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:43:33.856242 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd63b5a5-c5ef-4d3e-83a7-3e2d14e7fb24 None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:43:33.861705 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868f63c3-e0c4-4135-8994-810ca50e5ea5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:43:33.862633 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868f63c3-e0c4-4135-8994-810ca50e5ea5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:43:33.862985 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868f63c3-e0c4-4135-8994-810ca50e5ea5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:43:33.863263 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868f63c3-e0c4-4135-8994-810ca50e5ea5 admin admin] PATH_INFO: `/v3/projects/1cd818aebfc649e8b6f46c64f0120c61` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:43:33.864522 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868f63c3-e0c4-4135-8994-810ca50e5ea5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1cd818aebfc649e8b6f46c64f0120c61)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:43:33.866548 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868f63c3-e0c4-4135-8994-810ca50e5ea5 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:43:33.956330 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-868f63c3-e0c4-4135-8994-810ca50e5ea5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1cd818aebfc649e8b6f46c64f0120c61', 'request_id': 'req-868f63c3-e0c4-4135-8994-810ca50e5ea5'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:43:33.962098 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-868f63c3-e0c4-4135-8994-810ca50e5ea5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1cd818aebfc649e8b6f46c64f0120c61', 'request_id': 'req-868f63c3-e0c4-4135-8994-810ca50e5ea5'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:43:33.968348 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-868f63c3-e0c4-4135-8994-810ca50e5ea5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1cd818aebfc649e8b6f46c64f0120c61', 'request_id': 'req-868f63c3-e0c4-4135-8994-810ca50e5ea5'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:43:33.981612 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-868f63c3-e0c4-4135-8994-810ca50e5ea5 admin admin] The token cache is being invalidate because project 1cd818aebfc649e8b6f46c64f0120c61 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81759) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:43:33.981900 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-868f63c3-e0c4-4135-8994-810ca50e5ea5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:43:33.982511 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-868f63c3-e0c4-4135-8994-810ca50e5ea5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:43:33.983982 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1157/2329] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 17:43:33 2022] DELETE /identity/v3/projects/1cd818aebfc649e8b6f46c64f0120c61 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:43:38.953307 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a258d0-d207-40a8-92c9-dd1ad1302e2b None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:43:38.953782 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a258d0-d207-40a8-92c9-dd1ad1302e2b None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:43:38.953892 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a258d0-d207-40a8-92c9-dd1ad1302e2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:43:38.973846 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-72a258d0-d207-40a8-92c9-dd1ad1302e2b None None] MFA Rules not processed for user `ea143d042ee94ae7a4544d90d18db7db`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:43:38.975048 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-72a258d0-d207-40a8-92c9-dd1ad1302e2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:43:38.991510 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:43:39.034924 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1173/2330] 10.222.0.45 () {58 vars in 1009 bytes} [Wed Aug 24 17:43:38 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:43:39.047754 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868f63c3-e0c4-4135-8994-810ca50e5ea5 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:43:39.049158 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-868f63c3-e0c4-4135-8994-810ca50e5ea5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:43:39.063227 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:43:39.118634 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868f63c3-e0c4-4135-8994-810ca50e5ea5 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:43:39.123629 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af807e3e-b37c-4d7e-ad16-6644fcba6525 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90f0772b64c84faaa689e7272fa97898', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:43:39.124234 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af807e3e-b37c-4d7e-ad16-6644fcba6525 service swift] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:43:39.124514 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af807e3e-b37c-4d7e-ad16-6644fcba6525 service swift] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:43:39.124820 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af807e3e-b37c-4d7e-ad16-6644fcba6525 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:43:39.132852 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af807e3e-b37c-4d7e-ad16-6644fcba6525 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:43:39.134438 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af807e3e-b37c-4d7e-ad16-6644fcba6525 service swift] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:43:39.142421 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1158/2331] 10.222.0.45 () {60 vars in 1384 bytes} [Wed Aug 24 17:43:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 17:43:39.169125 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a258d0-d207-40a8-92c9-dd1ad1302e2b None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:43:39.170585 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-72a258d0-d207-40a8-92c9-dd1ad1302e2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:43:39.243464 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a258d0-d207-40a8-92c9-dd1ad1302e2b None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:43:39.249618 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d127b1b-5e05-4aad-a409-393fa368ba15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:43:39.250253 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d127b1b-5e05-4aad-a409-393fa368ba15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:43:39.250686 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d127b1b-5e05-4aad-a409-393fa368ba15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:43:39.250913 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d127b1b-5e05-4aad-a409-393fa368ba15 admin admin] PATH_INFO: `/v3/users/09c6006c56be4e89a4b6d51f3a74f36c` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:43:39.251982 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d127b1b-5e05-4aad-a409-393fa368ba15 admin admin] RBAC: Authorizing `identity:delete_user(user_id=09c6006c56be4e89a4b6d51f3a74f36c)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:43:39.254453 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d127b1b-5e05-4aad-a409-393fa368ba15 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:43:39.314610 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-9d127b1b-5e05-4aad-a409-393fa368ba15 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '09c6006c56be4e89a4b6d51f3a74f36c'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:43:39.316048 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-9d127b1b-5e05-4aad-a409-393fa368ba15 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '09c6006c56be4e89a4b6d51f3a74f36c'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:43:39.326850 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-9d127b1b-5e05-4aad-a409-393fa368ba15 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '09c6006c56be4e89a4b6d51f3a74f36c'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:43:39.327571 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-9d127b1b-5e05-4aad-a409-393fa368ba15 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '09c6006c56be4e89a4b6d51f3a74f36c'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:43:39.341245 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-9d127b1b-5e05-4aad-a409-393fa368ba15 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '09c6006c56be4e89a4b6d51f3a74f36c'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:43:39.360476 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1174/2332] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 17:43:39 2022] DELETE /identity/v3/users/09c6006c56be4e89a4b6d51f3a74f36c => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:43:39.363355 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c9cc6f-1a98-4b18-9987-be0123b7e7e3 None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:43:39.363689 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c9cc6f-1a98-4b18-9987-be0123b7e7e3 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:43:39.363947 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c9cc6f-1a98-4b18-9987-be0123b7e7e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:43:39.387890 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-b9c9cc6f-1a98-4b18-9987-be0123b7e7e3 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:43:39.389408 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-b9c9cc6f-1a98-4b18-9987-be0123b7e7e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:43:39.460633 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1159/2333] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:43:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:43:39.469299 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d127b1b-5e05-4aad-a409-393fa368ba15 admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:43:39.470967 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-9d127b1b-5e05-4aad-a409-393fa368ba15 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:43:39.535238 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:43:39.539907 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d127b1b-5e05-4aad-a409-393fa368ba15 admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:43:39.545107 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3c1c4b-5806-48bc-a722-3041b774bd5d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43ad98a250a144099ccd18378aedd4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:43:39.546349 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3c1c4b-5806-48bc-a722-3041b774bd5d service neutron] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:43:39.546868 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3c1c4b-5806-48bc-a722-3041b774bd5d service neutron] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:43:39.547254 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3c1c4b-5806-48bc-a722-3041b774bd5d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:43:39.558546 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3c1c4b-5806-48bc-a722-3041b774bd5d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:43:39.560241 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3c1c4b-5806-48bc-a722-3041b774bd5d service neutron] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:43:39.572133 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1175/2334] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:43:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:43:39.909146 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c9cc6f-1a98-4b18-9987-be0123b7e7e3 None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:43:39.911133 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-b9c9cc6f-1a98-4b18-9987-be0123b7e7e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:43:39.943304 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c9cc6f-1a98-4b18-9987-be0123b7e7e3 None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:43:39.955516 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06578f9-8278-4590-b166-1f57c724d8af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:43:39.956216 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06578f9-8278-4590-b166-1f57c724d8af admin admin] REQUEST_METHOD: `DELETE` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:43:39.956505 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06578f9-8278-4590-b166-1f57c724d8af admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:43:39.956897 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06578f9-8278-4590-b166-1f57c724d8af admin admin] PATH_INFO: `/v3/projects/a80f77d1ee7c4226aaf6390614475f65` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:43:39.957944 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06578f9-8278-4590-b166-1f57c724d8af admin admin] RBAC: Authorizing `identity:delete_project(project_id=a80f77d1ee7c4226aaf6390614475f65)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:43:39.959884 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06578f9-8278-4590-b166-1f57c724d8af admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:43:40.031571 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-b06578f9-8278-4590-b166-1f57c724d8af admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a80f77d1ee7c4226aaf6390614475f65', 'request_id': 'req-b06578f9-8278-4590-b166-1f57c724d8af'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:43:40.039280 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-b06578f9-8278-4590-b166-1f57c724d8af admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a80f77d1ee7c4226aaf6390614475f65', 'request_id': 'req-b06578f9-8278-4590-b166-1f57c724d8af'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:43:40.046397 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-b06578f9-8278-4590-b166-1f57c724d8af admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a80f77d1ee7c4226aaf6390614475f65', 'request_id': 'req-b06578f9-8278-4590-b166-1f57c724d8af'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:43:40.065039 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-b06578f9-8278-4590-b166-1f57c724d8af admin admin] The token cache is being invalidate because project a80f77d1ee7c4226aaf6390614475f65 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81759) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:43:40.065697 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-b06578f9-8278-4590-b166-1f57c724d8af admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:43:40.066176 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-b06578f9-8278-4590-b166-1f57c724d8af admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:43:40.068372 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1160/2335] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 17:43:39 2022] DELETE /identity/v3/projects/a80f77d1ee7c4226aaf6390614475f65 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:43:40.073835 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1588d042-e30a-4914-a1b8-a9d91874f5d8 None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:43:40.074664 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1588d042-e30a-4914-a1b8-a9d91874f5d8 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:43:40.075019 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1588d042-e30a-4914-a1b8-a9d91874f5d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:43:40.095194 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-1588d042-e30a-4914-a1b8-a9d91874f5d8 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:43:40.096976 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-1588d042-e30a-4914-a1b8-a9d91874f5d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:43:40.161453 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1176/2336] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:43:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:43:40.169338 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f69573-cab6-4078-8579-617887f8f445 None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:43:40.169868 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f69573-cab6-4078-8579-617887f8f445 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:43:40.170163 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f69573-cab6-4078-8579-617887f8f445 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:43:40.196526 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-f6f69573-cab6-4078-8579-617887f8f445 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:43:40.197822 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-f6f69573-cab6-4078-8579-617887f8f445 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:43:40.206034 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1161/2337] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:43:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:43:40.210431 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1588d042-e30a-4914-a1b8-a9d91874f5d8 None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:43:40.220886 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1588d042-e30a-4914-a1b8-a9d91874f5d8 None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:43:40.226570 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc9ed97-1fde-41bc-b2d7-40cd63ecbf77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:43:40.227860 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc9ed97-1fde-41bc-b2d7-40cd63ecbf77 admin admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:43:40.227860 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc9ed97-1fde-41bc-b2d7-40cd63ecbf77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:43:40.227860 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc9ed97-1fde-41bc-b2d7-40cd63ecbf77 admin admin] PATH_INFO: `/v3/domains` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:43:40.228509 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc9ed97-1fde-41bc-b2d7-40cd63ecbf77 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:43:40.228971 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc9ed97-1fde-41bc-b2d7-40cd63ecbf77 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:43:40.230464 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc9ed97-1fde-41bc-b2d7-40cd63ecbf77 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:43:40.244262 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1177/2338] 10.222.0.45 () {58 vars in 1153 bytes} [Wed Aug 24 17:43:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:43:40.248782 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f69573-cab6-4078-8579-617887f8f445 None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:43:40.264067 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f69573-cab6-4078-8579-617887f8f445 None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:43:40.271039 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c4c50c-ef22-4d04-bfce-06c001ae8e29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:43:40.272285 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c4c50c-ef22-4d04-bfce-06c001ae8e29 admin admin] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:43:40.272612 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c4c50c-ef22-4d04-bfce-06c001ae8e29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:43:40.272924 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c4c50c-ef22-4d04-bfce-06c001ae8e29 admin admin] PATH_INFO: `/v3/projects` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:43:40.273670 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c4c50c-ef22-4d04-bfce-06c001ae8e29 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:43:40.276204 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c4c50c-ef22-4d04-bfce-06c001ae8e29 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:43:40.302076 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:43:40.316870 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1162/2339] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 17:43:40 2022] POST /identity/v3/projects => generated 375 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:43:40.318397 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc9ed97-1fde-41bc-b2d7-40cd63ecbf77 admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:43:40.331261 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc9ed97-1fde-41bc-b2d7-40cd63ecbf77 admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:43:40.397222 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8da7b8-ba4b-4130-b2b1-eb0bd7cfd2d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:43:40.398109 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8da7b8-ba4b-4130-b2b1-eb0bd7cfd2d9 admin admin] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:43:40.398549 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8da7b8-ba4b-4130-b2b1-eb0bd7cfd2d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:43:40.398993 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8da7b8-ba4b-4130-b2b1-eb0bd7cfd2d9 admin admin] PATH_INFO: `/v3/users` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:43:40.399624 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8da7b8-ba4b-4130-b2b1-eb0bd7cfd2d9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:43:40.401735 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8da7b8-ba4b-4130-b2b1-eb0bd7cfd2d9 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:43:40.425760 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1178/2340] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 17:43:40 2022] POST /identity/v3/users => generated 288 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:43:40.429760 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c4c50c-ef22-4d04-bfce-06c001ae8e29 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:43:40.441461 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c4c50c-ef22-4d04-bfce-06c001ae8e29 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:43:40.458123 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae945bff-c94f-480c-b3a5-63a5cdbfc3b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:43:40.458123 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae945bff-c94f-480c-b3a5-63a5cdbfc3b1 admin admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:43:40.458123 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae945bff-c94f-480c-b3a5-63a5cdbfc3b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:43:40.458123 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae945bff-c94f-480c-b3a5-63a5cdbfc3b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:43:40.458722 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae945bff-c94f-480c-b3a5-63a5cdbfc3b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:43:40.458722 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae945bff-c94f-480c-b3a5-63a5cdbfc3b1 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:43:40.480603 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1163/2341] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:43:40 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 17:43:40.482128 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8da7b8-ba4b-4130-b2b1-eb0bd7cfd2d9 admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:43:40.493635 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8da7b8-ba4b-4130-b2b1-eb0bd7cfd2d9 admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:43:40.507463 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fcc41f-c218-44ab-a771-4f34be8e9377 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:43:40.508617 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fcc41f-c218-44ab-a771-4f34be8e9377 admin admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:43:40.510333 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fcc41f-c218-44ab-a771-4f34be8e9377 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:43:40.510803 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fcc41f-c218-44ab-a771-4f34be8e9377 admin admin] PATH_INFO: `/v3/roles` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:43:40.512977 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fcc41f-c218-44ab-a771-4f34be8e9377 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:43:40.514809 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fcc41f-c218-44ab-a771-4f34be8e9377 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:43:40.534804 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1179/2342] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:43:40 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:43:40.539265 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae945bff-c94f-480c-b3a5-63a5cdbfc3b1 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:43:40.565733 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae945bff-c94f-480c-b3a5-63a5cdbfc3b1 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:43:40.577935 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a63ae1f-9e1e-46dc-88ec-fd041dbd9960 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:43:40.579290 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a63ae1f-9e1e-46dc-88ec-fd041dbd9960 admin admin] REQUEST_METHOD: `PUT` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:43:40.579588 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a63ae1f-9e1e-46dc-88ec-fd041dbd9960 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:43:40.579937 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a63ae1f-9e1e-46dc-88ec-fd041dbd9960 admin admin] PATH_INFO: `/v3/projects/e702ac5e8a024e35a3053832d159eac7/users/259b5f9b8a8843eeafc36798ba3b6c58/roles/02476a5b82eb4b188c18a5d706a3ef68` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:43:40.614276 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a63ae1f-9e1e-46dc-88ec-fd041dbd9960 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e702ac5e8a024e35a3053832d159eac7, user_id=259b5f9b8a8843eeafc36798ba3b6c58, role_id=02476a5b82eb4b188c18a5d706a3ef68)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:43:40.617172 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a63ae1f-9e1e-46dc-88ec-fd041dbd9960 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:43:40.636706 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1164/2343] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 17:43:40 2022] PUT /identity/v3/projects/e702ac5e8a024e35a3053832d159eac7/users/259b5f9b8a8843eeafc36798ba3b6c58/roles/02476a5b82eb4b188c18a5d706a3ef68 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:43:40.647835 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a85b635-7caf-4fd9-9ad3-e8e8992e03ca None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:43:40.648545 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a85b635-7caf-4fd9-9ad3-e8e8992e03ca None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:43:40.648983 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a85b635-7caf-4fd9-9ad3-e8e8992e03ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:43:40.681018 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-7a85b635-7caf-4fd9-9ad3-e8e8992e03ca None None] MFA Rules not processed for user `259b5f9b8a8843eeafc36798ba3b6c58`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:43:40.682236 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-7a85b635-7caf-4fd9-9ad3-e8e8992e03ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:43:40.732494 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:43:40.755675 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1180/2344] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:43:40 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:43:40.767474 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a63ae1f-9e1e-46dc-88ec-fd041dbd9960 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:43:40.770703 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-4a63ae1f-9e1e-46dc-88ec-fd041dbd9960 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:43:40.839322 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a63ae1f-9e1e-46dc-88ec-fd041dbd9960 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:43:40.843182 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9336ba-4840-4a88-8b6e-efe1f2798647 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59be342541264942ad87f60ff4de6c8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:43:40.843870 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9336ba-4840-4a88-8b6e-efe1f2798647 service cinder] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:43:40.844223 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9336ba-4840-4a88-8b6e-efe1f2798647 service cinder] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:43:40.844618 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9336ba-4840-4a88-8b6e-efe1f2798647 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:43:40.854318 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9336ba-4840-4a88-8b6e-efe1f2798647 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:43:40.856013 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9336ba-4840-4a88-8b6e-efe1f2798647 service cinder] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:43:40.866448 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1165/2345] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:43:40 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:43:50.752195 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=7 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:43:50.862509 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=42,keystone:INSERT=1 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:43:51.653695 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a85b635-7caf-4fd9-9ad3-e8e8992e03ca None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:43:51.736647 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a85b635-7caf-4fd9-9ad3-e8e8992e03ca None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:43:51.741421 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3464198c-8c75-4174-bf74-24b87bf078a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:43:51.742125 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3464198c-8c75-4174-bf74-24b87bf078a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:43:51.742417 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3464198c-8c75-4174-bf74-24b87bf078a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:43:51.742696 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3464198c-8c75-4174-bf74-24b87bf078a0 admin admin] PATH_INFO: `/v3/users/259b5f9b8a8843eeafc36798ba3b6c58` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:43:51.744082 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3464198c-8c75-4174-bf74-24b87bf078a0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=259b5f9b8a8843eeafc36798ba3b6c58)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:43:51.746055 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3464198c-8c75-4174-bf74-24b87bf078a0 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:43:51.810478 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-3464198c-8c75-4174-bf74-24b87bf078a0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '259b5f9b8a8843eeafc36798ba3b6c58'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:43:51.811269 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-3464198c-8c75-4174-bf74-24b87bf078a0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '259b5f9b8a8843eeafc36798ba3b6c58'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:43:51.819151 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-3464198c-8c75-4174-bf74-24b87bf078a0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '259b5f9b8a8843eeafc36798ba3b6c58'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:43:51.819938 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-3464198c-8c75-4174-bf74-24b87bf078a0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '259b5f9b8a8843eeafc36798ba3b6c58'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:43:51.838469 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-3464198c-8c75-4174-bf74-24b87bf078a0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '259b5f9b8a8843eeafc36798ba3b6c58'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:43:51.861455 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1181/2346] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 17:43:51 2022] DELETE /identity/v3/users/259b5f9b8a8843eeafc36798ba3b6c58 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:43:51.863740 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b6ea10-a823-4296-9a55-f5efd755646d None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:43:51.864243 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b6ea10-a823-4296-9a55-f5efd755646d None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:43:51.864339 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b6ea10-a823-4296-9a55-f5efd755646d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:43:51.890528 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-41b6ea10-a823-4296-9a55-f5efd755646d None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:43:51.892130 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-41b6ea10-a823-4296-9a55-f5efd755646d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:43:51.969912 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1166/2347] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:43:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:43:51.982910 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3464198c-8c75-4174-bf74-24b87bf078a0 admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:43:51.984479 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-3464198c-8c75-4174-bf74-24b87bf078a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:43:52.078741 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3464198c-8c75-4174-bf74-24b87bf078a0 admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:43:52.083607 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c7c3aa-6f6d-409d-90cc-8e76aefdd79a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43ad98a250a144099ccd18378aedd4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:43:52.084220 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c7c3aa-6f6d-409d-90cc-8e76aefdd79a service neutron] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:43:52.084495 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c7c3aa-6f6d-409d-90cc-8e76aefdd79a service neutron] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:43:52.084834 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c7c3aa-6f6d-409d-90cc-8e76aefdd79a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:43:52.094250 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c7c3aa-6f6d-409d-90cc-8e76aefdd79a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:43:52.098479 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c7c3aa-6f6d-409d-90cc-8e76aefdd79a service neutron] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:43:52.109635 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1182/2348] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:43:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:43:52.421563 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b6ea10-a823-4296-9a55-f5efd755646d None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:43:52.423577 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-41b6ea10-a823-4296-9a55-f5efd755646d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:43:52.447572 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b6ea10-a823-4296-9a55-f5efd755646d None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:43:52.453961 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac748e02-3918-49b9-a013-4fabe611add9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:43:52.455092 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac748e02-3918-49b9-a013-4fabe611add9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:43:52.455092 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac748e02-3918-49b9-a013-4fabe611add9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:43:52.455282 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac748e02-3918-49b9-a013-4fabe611add9 admin admin] PATH_INFO: `/v3/projects/e702ac5e8a024e35a3053832d159eac7` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:43:52.456391 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac748e02-3918-49b9-a013-4fabe611add9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e702ac5e8a024e35a3053832d159eac7)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:43:52.458938 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac748e02-3918-49b9-a013-4fabe611add9 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:43:52.533554 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-ac748e02-3918-49b9-a013-4fabe611add9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e702ac5e8a024e35a3053832d159eac7', 'request_id': 'req-ac748e02-3918-49b9-a013-4fabe611add9'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:43:52.539955 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-ac748e02-3918-49b9-a013-4fabe611add9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e702ac5e8a024e35a3053832d159eac7', 'request_id': 'req-ac748e02-3918-49b9-a013-4fabe611add9'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:43:52.548490 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-ac748e02-3918-49b9-a013-4fabe611add9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e702ac5e8a024e35a3053832d159eac7', 'request_id': 'req-ac748e02-3918-49b9-a013-4fabe611add9'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:43:52.566231 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-ac748e02-3918-49b9-a013-4fabe611add9 admin admin] The token cache is being invalidate because project e702ac5e8a024e35a3053832d159eac7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81759) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:43:52.566664 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-ac748e02-3918-49b9-a013-4fabe611add9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:43:52.567388 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-ac748e02-3918-49b9-a013-4fabe611add9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:43:52.569248 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1167/2349] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 17:43:52 2022] DELETE /identity/v3/projects/e702ac5e8a024e35a3053832d159eac7 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:43:54.366742 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9382d8d-c250-47c3-ae69-7e8e2ac527d2 None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:43:54.367268 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9382d8d-c250-47c3-ae69-7e8e2ac527d2 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:43:54.367360 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9382d8d-c250-47c3-ae69-7e8e2ac527d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:43:54.386763 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-e9382d8d-c250-47c3-ae69-7e8e2ac527d2 None None] MFA Rules not processed for user `ea143d042ee94ae7a4544d90d18db7db`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:43:54.387884 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-e9382d8d-c250-47c3-ae69-7e8e2ac527d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:43:54.405342 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:43:54.443135 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1183/2350] 10.222.0.45 () {58 vars in 967 bytes} [Wed Aug 24 17:43:54 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:43:54.455421 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac748e02-3918-49b9-a013-4fabe611add9 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:43:54.457579 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-ac748e02-3918-49b9-a013-4fabe611add9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:43:54.525939 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac748e02-3918-49b9-a013-4fabe611add9 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:43:54.530936 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3c7b11-d514-4f9c-9b56-6bfa319ad6c3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90f0772b64c84faaa689e7272fa97898', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:43:54.531540 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3c7b11-d514-4f9c-9b56-6bfa319ad6c3 service swift] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:43:54.531820 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3c7b11-d514-4f9c-9b56-6bfa319ad6c3 service swift] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:43:54.532095 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3c7b11-d514-4f9c-9b56-6bfa319ad6c3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:43:54.539955 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3c7b11-d514-4f9c-9b56-6bfa319ad6c3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:43:54.541566 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3c7b11-d514-4f9c-9b56-6bfa319ad6c3 service swift] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:43:54.549463 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1168/2351] 10.222.0.45 () {60 vars in 1384 bytes} [Wed Aug 24 17:43:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 17:43:56.453464 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9382d8d-c250-47c3-ae69-7e8e2ac527d2 None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:43:56.454938 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-e9382d8d-c250-47c3-ae69-7e8e2ac527d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:43:56.518586 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9382d8d-c250-47c3-ae69-7e8e2ac527d2 None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:43:56.523286 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcce92c-a3bf-4c66-b172-d8d3fc844bfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:43:56.523906 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcce92c-a3bf-4c66-b172-d8d3fc844bfa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:43:56.524182 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcce92c-a3bf-4c66-b172-d8d3fc844bfa admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:43:56.524456 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcce92c-a3bf-4c66-b172-d8d3fc844bfa admin admin] PATH_INFO: `/v3/users/757f4cb8bb714ccc90b2f2589f9b2a1e` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:43:56.525593 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcce92c-a3bf-4c66-b172-d8d3fc844bfa admin admin] RBAC: Authorizing `identity:delete_user(user_id=757f4cb8bb714ccc90b2f2589f9b2a1e)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:43:56.527495 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcce92c-a3bf-4c66-b172-d8d3fc844bfa admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:43:56.573632 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-3fcce92c-a3bf-4c66-b172-d8d3fc844bfa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '757f4cb8bb714ccc90b2f2589f9b2a1e'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:43:56.574331 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-3fcce92c-a3bf-4c66-b172-d8d3fc844bfa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '757f4cb8bb714ccc90b2f2589f9b2a1e'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:43:56.581090 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-3fcce92c-a3bf-4c66-b172-d8d3fc844bfa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '757f4cb8bb714ccc90b2f2589f9b2a1e'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:43:56.581751 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-3fcce92c-a3bf-4c66-b172-d8d3fc844bfa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '757f4cb8bb714ccc90b2f2589f9b2a1e'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:43:56.594683 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-3fcce92c-a3bf-4c66-b172-d8d3fc844bfa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '757f4cb8bb714ccc90b2f2589f9b2a1e'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:43:56.608437 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1184/2352] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 17:43:56 2022] DELETE /identity/v3/users/757f4cb8bb714ccc90b2f2589f9b2a1e => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:43:56.774090 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3c7b11-d514-4f9c-9b56-6bfa319ad6c3 service swift] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:43:56.786369 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3c7b11-d514-4f9c-9b56-6bfa319ad6c3 service swift] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:43:56.797337 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:43:56.853300 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cccec96-1639-4223-ae9a-9dc99e0011e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:43:56.853859 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cccec96-1639-4223-ae9a-9dc99e0011e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:43:56.854166 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cccec96-1639-4223-ae9a-9dc99e0011e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:43:56.854437 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cccec96-1639-4223-ae9a-9dc99e0011e1 admin admin] PATH_INFO: `/v3/projects/8f5f06c64c9a459fa45ba85855d4324f` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:43:56.855555 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cccec96-1639-4223-ae9a-9dc99e0011e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8f5f06c64c9a459fa45ba85855d4324f)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:43:56.857526 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cccec96-1639-4223-ae9a-9dc99e0011e1 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:43:56.906594 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-4cccec96-1639-4223-ae9a-9dc99e0011e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8f5f06c64c9a459fa45ba85855d4324f', 'request_id': 'req-4cccec96-1639-4223-ae9a-9dc99e0011e1'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:43:56.912621 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-4cccec96-1639-4223-ae9a-9dc99e0011e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8f5f06c64c9a459fa45ba85855d4324f', 'request_id': 'req-4cccec96-1639-4223-ae9a-9dc99e0011e1'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:43:56.918665 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-4cccec96-1639-4223-ae9a-9dc99e0011e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8f5f06c64c9a459fa45ba85855d4324f', 'request_id': 'req-4cccec96-1639-4223-ae9a-9dc99e0011e1'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:43:56.930588 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-4cccec96-1639-4223-ae9a-9dc99e0011e1 admin admin] The token cache is being invalidate because project 8f5f06c64c9a459fa45ba85855d4324f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81759) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:43:56.930876 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-4cccec96-1639-4223-ae9a-9dc99e0011e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:43:56.931492 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-4cccec96-1639-4223-ae9a-9dc99e0011e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:43:56.933004 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1169/2353] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 17:43:56 2022] DELETE /identity/v3/projects/8f5f06c64c9a459fa45ba85855d4324f => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:43:56.938937 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc83be4-c7e0-4b6c-b227-59114159a48e None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:43:56.939223 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc83be4-c7e0-4b6c-b227-59114159a48e None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:43:56.939497 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc83be4-c7e0-4b6c-b227-59114159a48e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:43:56.959006 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-efc83be4-c7e0-4b6c-b227-59114159a48e None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:43:56.960202 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-efc83be4-c7e0-4b6c-b227-59114159a48e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:43:57.000669 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:43:57.020132 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1185/2354] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:43:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:43:57.027594 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0f7152-167d-4ee8-9b7c-628691ceb18e None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:43:57.027885 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0f7152-167d-4ee8-9b7c-628691ceb18e None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:43:57.028166 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0f7152-167d-4ee8-9b7c-628691ceb18e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:43:57.047904 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-2b0f7152-167d-4ee8-9b7c-628691ceb18e None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:43:57.049143 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-2b0f7152-167d-4ee8-9b7c-628691ceb18e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:43:57.057224 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1170/2355] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:43:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:43:57.061074 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc83be4-c7e0-4b6c-b227-59114159a48e None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:43:57.070439 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc83be4-c7e0-4b6c-b227-59114159a48e None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:43:57.074997 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf47983-e183-441f-89e0-88b137f37c28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:43:57.075588 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf47983-e183-441f-89e0-88b137f37c28 admin admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:43:57.075867 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf47983-e183-441f-89e0-88b137f37c28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:43:57.076138 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf47983-e183-441f-89e0-88b137f37c28 admin admin] PATH_INFO: `/v3/domains` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:43:57.076815 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf47983-e183-441f-89e0-88b137f37c28 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:43:57.077117 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf47983-e183-441f-89e0-88b137f37c28 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:43:57.078700 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf47983-e183-441f-89e0-88b137f37c28 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:43:57.090484 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1186/2356] 10.222.0.45 () {58 vars in 1153 bytes} [Wed Aug 24 17:43:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:43:57.094505 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0f7152-167d-4ee8-9b7c-628691ceb18e None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:43:57.104275 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0f7152-167d-4ee8-9b7c-628691ceb18e None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:43:57.108824 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955a8c8c-74d8-40c5-bcf9-45539b6de3fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:43:57.109412 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955a8c8c-74d8-40c5-bcf9-45539b6de3fa admin admin] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:43:57.109688 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955a8c8c-74d8-40c5-bcf9-45539b6de3fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:43:57.109959 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955a8c8c-74d8-40c5-bcf9-45539b6de3fa admin admin] PATH_INFO: `/v3/projects` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:43:57.110581 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955a8c8c-74d8-40c5-bcf9-45539b6de3fa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:43:57.112464 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955a8c8c-74d8-40c5-bcf9-45539b6de3fa admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:43:57.138932 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1171/2357] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 17:43:57 2022] POST /identity/v3/projects => generated 371 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:43:57.142810 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf47983-e183-441f-89e0-88b137f37c28 admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:43:57.153423 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf47983-e183-441f-89e0-88b137f37c28 admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:43:57.209840 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cc421d-a1df-48a0-a7e4-b5299a6a7a53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:43:57.210443 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cc421d-a1df-48a0-a7e4-b5299a6a7a53 admin admin] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:43:57.210726 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cc421d-a1df-48a0-a7e4-b5299a6a7a53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:43:57.211000 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cc421d-a1df-48a0-a7e4-b5299a6a7a53 admin admin] PATH_INFO: `/v3/users` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:43:57.211617 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cc421d-a1df-48a0-a7e4-b5299a6a7a53 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:43:57.213569 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cc421d-a1df-48a0-a7e4-b5299a6a7a53 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:43:57.235304 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1187/2358] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 17:43:57 2022] POST /identity/v3/users => generated 286 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:43:57.239925 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955a8c8c-74d8-40c5-bcf9-45539b6de3fa admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:43:57.250173 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955a8c8c-74d8-40c5-bcf9-45539b6de3fa admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:43:57.254987 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee4789c-409b-480f-9df0-d002bdd74505 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:43:57.255578 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee4789c-409b-480f-9df0-d002bdd74505 admin admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:43:57.255854 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee4789c-409b-480f-9df0-d002bdd74505 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:43:57.256150 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee4789c-409b-480f-9df0-d002bdd74505 admin admin] PATH_INFO: `/v3/roles` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:43:57.256837 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee4789c-409b-480f-9df0-d002bdd74505 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:43:57.258532 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee4789c-409b-480f-9df0-d002bdd74505 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:43:57.269214 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1172/2359] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:43:57 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:43:57.273224 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cc421d-a1df-48a0-a7e4-b5299a6a7a53 admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:43:57.282774 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cc421d-a1df-48a0-a7e4-b5299a6a7a53 admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:43:57.287837 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe7ec69-4fcb-4432-90cb-cd428aae7c53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:43:57.288424 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe7ec69-4fcb-4432-90cb-cd428aae7c53 admin admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:43:57.288731 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe7ec69-4fcb-4432-90cb-cd428aae7c53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:43:57.289006 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe7ec69-4fcb-4432-90cb-cd428aae7c53 admin admin] PATH_INFO: `/v3/roles` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:43:57.289642 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe7ec69-4fcb-4432-90cb-cd428aae7c53 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:43:57.291206 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe7ec69-4fcb-4432-90cb-cd428aae7c53 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:43:57.302409 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1188/2360] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:43:57 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 17:43:57.306227 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee4789c-409b-480f-9df0-d002bdd74505 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:43:57.315916 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee4789c-409b-480f-9df0-d002bdd74505 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:43:57.320396 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfedd40-4984-4af8-9b8e-a8dc22bbabb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:43:57.321128 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfedd40-4984-4af8-9b8e-a8dc22bbabb4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:43:57.321408 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfedd40-4984-4af8-9b8e-a8dc22bbabb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:43:57.321678 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfedd40-4984-4af8-9b8e-a8dc22bbabb4 admin admin] PATH_INFO: `/v3/projects/f45fcc8295e24580b04787b4040d3612/users/8d38f974f55d4f27bafdfe06ac9dfebb/roles/02476a5b82eb4b188c18a5d706a3ef68` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:43:57.335706 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfedd40-4984-4af8-9b8e-a8dc22bbabb4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f45fcc8295e24580b04787b4040d3612, user_id=8d38f974f55d4f27bafdfe06ac9dfebb, role_id=02476a5b82eb4b188c18a5d706a3ef68)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:43:57.338372 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfedd40-4984-4af8-9b8e-a8dc22bbabb4 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:43:57.351236 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1173/2361] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 17:43:57 2022] PUT /identity/v3/projects/f45fcc8295e24580b04787b4040d3612/users/8d38f974f55d4f27bafdfe06ac9dfebb/roles/02476a5b82eb4b188c18a5d706a3ef68 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:43:57.356182 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a6775d-8f10-4547-8939-184eee184c58 None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:43:57.356411 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a6775d-8f10-4547-8939-184eee184c58 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:43:57.356715 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a6775d-8f10-4547-8939-184eee184c58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:43:57.375527 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-c9a6775d-8f10-4547-8939-184eee184c58 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:43:57.376781 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-c9a6775d-8f10-4547-8939-184eee184c58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:43:57.433018 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1189/2362] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:43:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:43:57.442547 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfedd40-4984-4af8-9b8e-a8dc22bbabb4 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:43:57.444237 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-0dfedd40-4984-4af8-9b8e-a8dc22bbabb4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:43:57.486463 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:43:57.502166 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfedd40-4984-4af8-9b8e-a8dc22bbabb4 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:43:57.506534 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057d61f4-e148-4ab7-b55f-97dc155beb20 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43ad98a250a144099ccd18378aedd4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:43:57.507212 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057d61f4-e148-4ab7-b55f-97dc155beb20 service neutron] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:43:57.507562 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057d61f4-e148-4ab7-b55f-97dc155beb20 service neutron] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:43:57.507899 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057d61f4-e148-4ab7-b55f-97dc155beb20 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:43:57.517370 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057d61f4-e148-4ab7-b55f-97dc155beb20 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:43:57.518956 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057d61f4-e148-4ab7-b55f-97dc155beb20 service neutron] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:43:57.528915 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1174/2363] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:43:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:44:01.287785 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578c3657-727a-4d84-a607-31caf55378a7 None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:44:01.288517 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578c3657-727a-4d84-a607-31caf55378a7 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:44:01.288517 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578c3657-727a-4d84-a607-31caf55378a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:44:01.316386 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-578c3657-727a-4d84-a607-31caf55378a7 None None] MFA Rules not processed for user `8d38f974f55d4f27bafdfe06ac9dfebb`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:44:01.317631 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-578c3657-727a-4d84-a607-31caf55378a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:44:01.359855 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:44:01.384505 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1190/2364] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:44:01 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:44:01.389083 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057d61f4-e148-4ab7-b55f-97dc155beb20 service neutron] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:44:01.399495 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057d61f4-e148-4ab7-b55f-97dc155beb20 service neutron] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:44:01.404116 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f928143-6a7c-4294-93f3-a97d726879fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:44:01.404746 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f928143-6a7c-4294-93f3-a97d726879fb admin admin] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:44:01.405029 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f928143-6a7c-4294-93f3-a97d726879fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:44:01.405302 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f928143-6a7c-4294-93f3-a97d726879fb admin admin] PATH_INFO: `/v3/projects` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:44:01.405929 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f928143-6a7c-4294-93f3-a97d726879fb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:44:01.407841 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f928143-6a7c-4294-93f3-a97d726879fb admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:44:01.435361 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1175/2365] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 17:44:01 2022] POST /identity/v3/projects => generated 371 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:44:01.439365 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578c3657-727a-4d84-a607-31caf55378a7 None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:44:01.450131 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578c3657-727a-4d84-a607-31caf55378a7 None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:44:01.519090 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0b0c3e-6ae5-4f89-990c-a293886ce5a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:44:01.519958 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0b0c3e-6ae5-4f89-990c-a293886ce5a2 admin admin] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:44:01.520238 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0b0c3e-6ae5-4f89-990c-a293886ce5a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:44:01.520520 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0b0c3e-6ae5-4f89-990c-a293886ce5a2 admin admin] PATH_INFO: `/v3/users` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:44:01.521185 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0b0c3e-6ae5-4f89-990c-a293886ce5a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:44:01.523111 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0b0c3e-6ae5-4f89-990c-a293886ce5a2 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:44:01.547126 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1191/2366] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 17:44:01 2022] POST /identity/v3/users => generated 292 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:44:01.554543 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f928143-6a7c-4294-93f3-a97d726879fb admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:44:01.565523 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f928143-6a7c-4294-93f3-a97d726879fb admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:44:01.571128 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8095a5d-dfe5-455d-8ede-8fdd96bc6faf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:44:01.571818 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8095a5d-dfe5-455d-8ede-8fdd96bc6faf admin admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:44:01.572165 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8095a5d-dfe5-455d-8ede-8fdd96bc6faf admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:44:01.572507 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8095a5d-dfe5-455d-8ede-8fdd96bc6faf admin admin] PATH_INFO: `/v3/roles` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:44:01.574866 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8095a5d-dfe5-455d-8ede-8fdd96bc6faf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:44:01.574990 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8095a5d-dfe5-455d-8ede-8fdd96bc6faf admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:44:01.588337 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1176/2367] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:44:01 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:44:01.592419 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0b0c3e-6ae5-4f89-990c-a293886ce5a2 admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:44:01.603338 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0b0c3e-6ae5-4f89-990c-a293886ce5a2 admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:44:01.609002 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b35717-59c8-4cc3-95ce-6c96713e7085 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:44:01.609839 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b35717-59c8-4cc3-95ce-6c96713e7085 admin admin] REQUEST_METHOD: `PUT` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:44:01.610275 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b35717-59c8-4cc3-95ce-6c96713e7085 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:44:01.610685 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b35717-59c8-4cc3-95ce-6c96713e7085 admin admin] PATH_INFO: `/v3/projects/61e6476063b748cbb69fb9b2bedfe34f/users/a1027018a8fd4b00bf27e2aed7d71204/roles/e411ad9eb96e4e8d85b59b5b700ada7b` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:44:01.641985 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b35717-59c8-4cc3-95ce-6c96713e7085 admin admin] RBAC: Authorizing `identity:create_grant(project_id=61e6476063b748cbb69fb9b2bedfe34f, user_id=a1027018a8fd4b00bf27e2aed7d71204, role_id=e411ad9eb96e4e8d85b59b5b700ada7b)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:44:01.647059 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b35717-59c8-4cc3-95ce-6c96713e7085 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:44:01.661805 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1192/2368] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 17:44:01 2022] PUT /identity/v3/projects/61e6476063b748cbb69fb9b2bedfe34f/users/a1027018a8fd4b00bf27e2aed7d71204/roles/e411ad9eb96e4e8d85b59b5b700ada7b => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:44:01.665689 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8095a5d-dfe5-455d-8ede-8fdd96bc6faf admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:44:01.675820 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8095a5d-dfe5-455d-8ede-8fdd96bc6faf admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:44:01.747025 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ea6a50-e6bc-4334-844f-0b42a011a805 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:44:01.747743 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ea6a50-e6bc-4334-844f-0b42a011a805 admin admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:44:01.748094 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ea6a50-e6bc-4334-844f-0b42a011a805 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:44:01.748470 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ea6a50-e6bc-4334-844f-0b42a011a805 admin admin] PATH_INFO: `/v3/roles` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:44:01.749285 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ea6a50-e6bc-4334-844f-0b42a011a805 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:44:01.751058 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ea6a50-e6bc-4334-844f-0b42a011a805 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:44:01.762618 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1177/2369] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:44:01 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 17:44:01.766770 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b35717-59c8-4cc3-95ce-6c96713e7085 admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:44:01.778507 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b35717-59c8-4cc3-95ce-6c96713e7085 admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:44:01.783324 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212bc6e8-34b3-4b4d-915d-08402f18ddfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:44:01.784034 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212bc6e8-34b3-4b4d-915d-08402f18ddfb admin admin] REQUEST_METHOD: `PUT` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:44:01.784311 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212bc6e8-34b3-4b4d-915d-08402f18ddfb admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:44:01.784619 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212bc6e8-34b3-4b4d-915d-08402f18ddfb admin admin] PATH_INFO: `/v3/projects/61e6476063b748cbb69fb9b2bedfe34f/users/a1027018a8fd4b00bf27e2aed7d71204/roles/e411ad9eb96e4e8d85b59b5b700ada7b` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:44:01.786566 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212bc6e8-34b3-4b4d-915d-08402f18ddfb admin admin] RBAC: Authorizing `identity:create_grant(project_id=61e6476063b748cbb69fb9b2bedfe34f, user_id=a1027018a8fd4b00bf27e2aed7d71204, role_id=e411ad9eb96e4e8d85b59b5b700ada7b)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:44:01.789272 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212bc6e8-34b3-4b4d-915d-08402f18ddfb admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:44:01.804979 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1193/2370] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 17:44:01 2022] PUT /identity/v3/projects/61e6476063b748cbb69fb9b2bedfe34f/users/a1027018a8fd4b00bf27e2aed7d71204/roles/e411ad9eb96e4e8d85b59b5b700ada7b => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:44:05.876311 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3931d4d0-d31f-4047-ae20-1c5d41c9a7d6 None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:44:05.876837 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3931d4d0-d31f-4047-ae20-1c5d41c9a7d6 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:44:05.876934 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3931d4d0-d31f-4047-ae20-1c5d41c9a7d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:44:05.913319 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-3931d4d0-d31f-4047-ae20-1c5d41c9a7d6 None None] MFA Rules not processed for user `a1027018a8fd4b00bf27e2aed7d71204`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:44:05.914595 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-3931d4d0-d31f-4047-ae20-1c5d41c9a7d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:44:06.003663 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1178/2371] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:44:05 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:44:06.022731 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212bc6e8-34b3-4b4d-915d-08402f18ddfb admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:44:06.025651 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-212bc6e8-34b3-4b4d-915d-08402f18ddfb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:44:06.133957 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212bc6e8-34b3-4b4d-915d-08402f18ddfb admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:44:06.140097 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85fc496-c555-4213-ae74-bf708aae07d7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '053b7ce6d6f649ed91a26e434dab805e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:44:06.141956 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85fc496-c555-4213-ae74-bf708aae07d7 service nova] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:44:06.142241 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85fc496-c555-4213-ae74-bf708aae07d7 service nova] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:44:06.142519 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85fc496-c555-4213-ae74-bf708aae07d7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:44:06.185519 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:44:06.213936 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85fc496-c555-4213-ae74-bf708aae07d7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:44:06.215572 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85fc496-c555-4213-ae74-bf708aae07d7 service nova] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:44:06.226329 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1194/2372] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:44:06 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:44:07.181893 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d96ff8-fbb8-4905-97ba-89a38c07b351 None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:44:07.182419 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d96ff8-fbb8-4905-97ba-89a38c07b351 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:44:07.182567 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d96ff8-fbb8-4905-97ba-89a38c07b351 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:44:07.202734 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-e7d96ff8-fbb8-4905-97ba-89a38c07b351 None None] MFA Rules not processed for user `8d38f974f55d4f27bafdfe06ac9dfebb`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:44:07.203941 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-e7d96ff8-fbb8-4905-97ba-89a38c07b351 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:44:07.211427 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1179/2373] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:44:07 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:44:16.221273 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=13 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:44:17.197137 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=88,keystone:INSERT=1 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:44:22.125270 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870a0e9f-fc8f-4025-866d-ad07a1eb401e None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:44:22.125768 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870a0e9f-fc8f-4025-866d-ad07a1eb401e None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:44:22.125870 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870a0e9f-fc8f-4025-866d-ad07a1eb401e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:44:22.145786 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-870a0e9f-fc8f-4025-866d-ad07a1eb401e None None] MFA Rules not processed for user `c36a9a0d92024734b1aa007a56c6328c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:44:22.147572 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-870a0e9f-fc8f-4025-866d-ad07a1eb401e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:44:22.201057 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1195/2374] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:44:22 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:44:32.193787 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=25 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:44:40.832844 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7069e185-3434-4364-b190-d468bf25b97d None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:44:40.833817 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7069e185-3434-4364-b190-d468bf25b97d None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:44:40.834247 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7069e185-3434-4364-b190-d468bf25b97d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:44:40.855716 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-7069e185-3434-4364-b190-d468bf25b97d None None] MFA Rules not processed for user `ea143d042ee94ae7a4544d90d18db7db`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:44:40.856743 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-7069e185-3434-4364-b190-d468bf25b97d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:44:40.926066 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1180/2375] 10.222.0.45 () {58 vars in 1009 bytes} [Wed Aug 24 17:44:40 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:44:40.942650 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870a0e9f-fc8f-4025-866d-ad07a1eb401e None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:44:40.945585 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-870a0e9f-fc8f-4025-866d-ad07a1eb401e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:44:41.032985 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870a0e9f-fc8f-4025-866d-ad07a1eb401e None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:44:41.038135 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d9bc5f-e53e-4b92-9b53-af63437fa4b1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90f0772b64c84faaa689e7272fa97898', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:44:41.039362 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d9bc5f-e53e-4b92-9b53-af63437fa4b1 service swift] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:44:41.039712 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d9bc5f-e53e-4b92-9b53-af63437fa4b1 service swift] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:44:41.040482 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d9bc5f-e53e-4b92-9b53-af63437fa4b1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:44:41.048819 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d9bc5f-e53e-4b92-9b53-af63437fa4b1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:44:41.050656 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d9bc5f-e53e-4b92-9b53-af63437fa4b1 service swift] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:44:41.060136 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1196/2376] 10.222.0.45 () {60 vars in 1384 bytes} [Wed Aug 24 17:44:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 17:44:50.915986 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=25 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:44:51.056941 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=32 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:44:52.080859 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7069e185-3434-4364-b190-d468bf25b97d None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:44:52.134711 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7069e185-3434-4364-b190-d468bf25b97d None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:44:52.138450 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a538fd84-1219-4eb3-9548-3913b47b7d24 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43ad98a250a144099ccd18378aedd4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:44:52.139066 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a538fd84-1219-4eb3-9548-3913b47b7d24 service neutron] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:44:52.139350 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a538fd84-1219-4eb3-9548-3913b47b7d24 service neutron] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:44:52.139628 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a538fd84-1219-4eb3-9548-3913b47b7d24 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:44:52.149673 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a538fd84-1219-4eb3-9548-3913b47b7d24 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:44:52.151350 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a538fd84-1219-4eb3-9548-3913b47b7d24 service neutron] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:44:52.162475 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1181/2377] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:44:52 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:44:55.854617 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da556b5-cea7-4d98-a44a-f435c759a826 None None] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:44:55.855196 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da556b5-cea7-4d98-a44a-f435c759a826 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:44:55.855196 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da556b5-cea7-4d98-a44a-f435c759a826 None None] PATH_INFO: `/` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:44:55.856568 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1197/2378] 10.222.0.45 () {54 vars in 921 bytes} [Wed Aug 24 17:44:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:44:55.863144 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dc04ce-76d5-42e8-a5c6-2e32a1213882 None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:44:55.864147 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dc04ce-76d5-42e8-a5c6-2e32a1213882 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:44:55.864671 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dc04ce-76d5-42e8-a5c6-2e32a1213882 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:44:55.885242 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-45dc04ce-76d5-42e8-a5c6-2e32a1213882 None None] MFA Rules not processed for user `053b7ce6d6f649ed91a26e434dab805e`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:44:55.886466 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-45dc04ce-76d5-42e8-a5c6-2e32a1213882 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:44:55.896532 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1182/2379] 10.222.0.45 () {58 vars in 1018 bytes} [Wed Aug 24 17:44:55 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:44:55.908456 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da556b5-cea7-4d98-a44a-f435c759a826 None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:44:55.910237 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-0da556b5-cea7-4d98-a44a-f435c759a826 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:44:55.969358 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da556b5-cea7-4d98-a44a-f435c759a826 None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:44:55.972953 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165dc28e-6dcc-483d-9a6f-c1a3e2bd162b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df5fee57539447428dd1c72eab3cfd9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:44:55.973565 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165dc28e-6dcc-483d-9a6f-c1a3e2bd162b service glance] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:44:55.973846 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165dc28e-6dcc-483d-9a6f-c1a3e2bd162b service glance] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:44:55.974121 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165dc28e-6dcc-483d-9a6f-c1a3e2bd162b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:44:55.985016 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165dc28e-6dcc-483d-9a6f-c1a3e2bd162b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:44:55.986449 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165dc28e-6dcc-483d-9a6f-c1a3e2bd162b service glance] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:44:55.997450 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1198/2380] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:44:55 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:44:56.064452 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071b2536-4c74-4e20-867f-18aeacaa996f None None] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:44:56.064770 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071b2536-4c74-4e20-867f-18aeacaa996f None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:44:56.065033 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071b2536-4c74-4e20-867f-18aeacaa996f None None] PATH_INFO: `/` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:44:56.066359 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1183/2381] 10.222.0.45 () {54 vars in 921 bytes} [Wed Aug 24 17:44:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:44:56.071575 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3a1058-053f-49b8-9a9a-b52e478e4302 None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:44:56.071861 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3a1058-053f-49b8-9a9a-b52e478e4302 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:44:56.072168 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3a1058-053f-49b8-9a9a-b52e478e4302 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:44:56.092499 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-4b3a1058-053f-49b8-9a9a-b52e478e4302 None None] MFA Rules not processed for user `43ad98a250a144099ccd18378aedd4b5`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:44:56.093741 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-4b3a1058-053f-49b8-9a9a-b52e478e4302 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:44:56.100755 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1199/2382] 10.222.0.45 () {58 vars in 1018 bytes} [Wed Aug 24 17:44:56 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:44:56.111995 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071b2536-4c74-4e20-867f-18aeacaa996f None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:44:56.123200 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071b2536-4c74-4e20-867f-18aeacaa996f None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:44:56.126633 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010006aa-ee89-4d52-88cd-92902b4beafb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43ad98a250a144099ccd18378aedd4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:44:56.127214 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010006aa-ee89-4d52-88cd-92902b4beafb service neutron] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:44:56.127490 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010006aa-ee89-4d52-88cd-92902b4beafb service neutron] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:44:56.127761 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010006aa-ee89-4d52-88cd-92902b4beafb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:44:56.135270 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010006aa-ee89-4d52-88cd-92902b4beafb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:44:56.136810 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010006aa-ee89-4d52-88cd-92902b4beafb service neutron] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:44:56.144567 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1184/2383] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:44:56 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 33 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 17:44:56.225988 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5f6c95-d3d4-41c2-9fc6-199a2318050e None None] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:44:56.226276 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5f6c95-d3d4-41c2-9fc6-199a2318050e None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:44:56.226550 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5f6c95-d3d4-41c2-9fc6-199a2318050e None None] PATH_INFO: `/` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:44:56.227895 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1200/2384] 10.222.0.45 () {54 vars in 921 bytes} [Wed Aug 24 17:44:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:44:56.233789 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3fba46-3636-43df-8cb8-31ff62e06eb4 None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:44:56.234072 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3fba46-3636-43df-8cb8-31ff62e06eb4 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:44:56.234346 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3fba46-3636-43df-8cb8-31ff62e06eb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:44:56.253273 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-8c3fba46-3636-43df-8cb8-31ff62e06eb4 None None] MFA Rules not processed for user `053b7ce6d6f649ed91a26e434dab805e`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:44:56.254457 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-8c3fba46-3636-43df-8cb8-31ff62e06eb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:44:56.264453 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1185/2385] 10.222.0.45 () {58 vars in 1018 bytes} [Wed Aug 24 17:44:56 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:44:56.275505 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5f6c95-d3d4-41c2-9fc6-199a2318050e None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:44:56.286380 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5f6c95-d3d4-41c2-9fc6-199a2318050e None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:44:56.289947 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3dd2d9-cf42-416a-b05b-77e81ac8bdfe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43ad98a250a144099ccd18378aedd4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:44:56.290539 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3dd2d9-cf42-416a-b05b-77e81ac8bdfe service neutron] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:44:56.290820 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3dd2d9-cf42-416a-b05b-77e81ac8bdfe service neutron] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:44:56.291095 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3dd2d9-cf42-416a-b05b-77e81ac8bdfe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:44:56.300899 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3dd2d9-cf42-416a-b05b-77e81ac8bdfe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:44:56.302332 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3dd2d9-cf42-416a-b05b-77e81ac8bdfe service neutron] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:44:56.312823 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1201/2386] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:44:56 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:44:58.690914 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dac8e3b-9b25-4d52-8e78-23b885986816 None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:44:58.691387 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dac8e3b-9b25-4d52-8e78-23b885986816 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:44:58.691482 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dac8e3b-9b25-4d52-8e78-23b885986816 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:44:58.710040 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-1dac8e3b-9b25-4d52-8e78-23b885986816 None None] MFA Rules not processed for user `ea143d042ee94ae7a4544d90d18db7db`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:44:58.711223 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-1dac8e3b-9b25-4d52-8e78-23b885986816 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:44:58.718930 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1186/2387] 10.222.0.45 () {58 vars in 1009 bytes} [Wed Aug 24 17:44:58 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:44:58.731205 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3dd2d9-cf42-416a-b05b-77e81ac8bdfe service neutron] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:44:58.742084 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3dd2d9-cf42-416a-b05b-77e81ac8bdfe service neutron] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:44:58.746968 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598c00b4-a805-41cc-8dca-cd9426282f5f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90f0772b64c84faaa689e7272fa97898', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:44:58.747555 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598c00b4-a805-41cc-8dca-cd9426282f5f service swift] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:44:58.747832 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598c00b4-a805-41cc-8dca-cd9426282f5f service swift] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:44:58.748143 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598c00b4-a805-41cc-8dca-cd9426282f5f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:44:58.755817 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598c00b4-a805-41cc-8dca-cd9426282f5f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:44:58.757441 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598c00b4-a805-41cc-8dca-cd9426282f5f service swift] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:44:58.765227 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1202/2388] 10.222.0.45 () {60 vars in 1384 bytes} [Wed Aug 24 17:44:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 17:45:08.704942 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=41 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:45:08.762193 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=41 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:45:12.521823 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dac8e3b-9b25-4d52-8e78-23b885986816 None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:45:12.523571 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-1dac8e3b-9b25-4d52-8e78-23b885986816 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:45:12.535578 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dac8e3b-9b25-4d52-8e78-23b885986816 None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:45:12.539112 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f90a426-b1ed-42ca-92f1-65285a4c3244 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43ad98a250a144099ccd18378aedd4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:45:12.539732 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f90a426-b1ed-42ca-92f1-65285a4c3244 service neutron] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:45:12.540012 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f90a426-b1ed-42ca-92f1-65285a4c3244 service neutron] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:45:12.540287 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f90a426-b1ed-42ca-92f1-65285a4c3244 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:45:12.541987 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-5f90a426-b1ed-42ca-92f1-65285a4c3244 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:45:12.555197 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f90a426-b1ed-42ca-92f1-65285a4c3244 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:45:12.556666 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f90a426-b1ed-42ca-92f1-65285a4c3244 service neutron] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:45:12.567492 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1187/2389] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:45:12 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:45:13.748888 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598c00b4-a805-41cc-8dca-cd9426282f5f service swift] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:45:13.757845 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598c00b4-a805-41cc-8dca-cd9426282f5f service swift] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:45:13.761411 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ee240f-9016-4853-9a8e-d802f88d7899 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43ad98a250a144099ccd18378aedd4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:45:13.761995 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ee240f-9016-4853-9a8e-d802f88d7899 service neutron] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:45:13.762271 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ee240f-9016-4853-9a8e-d802f88d7899 service neutron] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:45:13.762541 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ee240f-9016-4853-9a8e-d802f88d7899 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:45:13.764220 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-23ee240f-9016-4853-9a8e-d802f88d7899 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:45:13.778243 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ee240f-9016-4853-9a8e-d802f88d7899 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:45:13.779668 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ee240f-9016-4853-9a8e-d802f88d7899 service neutron] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:45:13.790900 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1203/2390] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:45:13 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:45:22.562667 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=6 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:45:22.564362 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f90a426-b1ed-42ca-92f1-65285a4c3244 service neutron] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:45:22.566212 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-5f90a426-b1ed-42ca-92f1-65285a4c3244 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:45:22.620157 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f90a426-b1ed-42ca-92f1-65285a4c3244 service neutron] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:45:22.623670 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c08b92-1fb6-4d50-a06b-c342122619ad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59be342541264942ad87f60ff4de6c8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:45:22.624260 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c08b92-1fb6-4d50-a06b-c342122619ad service cinder] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:45:22.624533 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c08b92-1fb6-4d50-a06b-c342122619ad service cinder] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:45:22.624834 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c08b92-1fb6-4d50-a06b-c342122619ad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:45:22.626489 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-61c08b92-1fb6-4d50-a06b-c342122619ad service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:45:22.688064 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c08b92-1fb6-4d50-a06b-c342122619ad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:45:22.689560 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c08b92-1fb6-4d50-a06b-c342122619ad service cinder] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:45:22.698381 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1188/2391] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:45:22 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:45:23.785777 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=6 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:45:28.484516 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda18b44-dedf-437e-9021-480c7b4cc7a1 None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:45:28.485111 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda18b44-dedf-437e-9021-480c7b4cc7a1 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:45:28.485111 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda18b44-dedf-437e-9021-480c7b4cc7a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:45:28.504067 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-fda18b44-dedf-437e-9021-480c7b4cc7a1 None None] MFA Rules not processed for user `ea143d042ee94ae7a4544d90d18db7db`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:45:28.505283 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-fda18b44-dedf-437e-9021-480c7b4cc7a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:45:28.512253 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1204/2392] 10.222.0.45 () {58 vars in 967 bytes} [Wed Aug 24 17:45:28 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:45:28.524607 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c08b92-1fb6-4d50-a06b-c342122619ad service cinder] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:45:28.535993 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c08b92-1fb6-4d50-a06b-c342122619ad service cinder] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:45:28.541510 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16e1bcc-f417-4640-89d8-4301a20520ca service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90f0772b64c84faaa689e7272fa97898', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:45:28.542111 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16e1bcc-f417-4640-89d8-4301a20520ca service swift] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:45:28.542394 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16e1bcc-f417-4640-89d8-4301a20520ca service swift] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:45:28.542670 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16e1bcc-f417-4640-89d8-4301a20520ca service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:45:28.551224 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16e1bcc-f417-4640-89d8-4301a20520ca service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:45:28.552996 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16e1bcc-f417-4640-89d8-4301a20520ca service swift] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:45:28.561631 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1189/2393] 10.222.0.45 () {60 vars in 1384 bytes} [Wed Aug 24 17:45:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 24 17:45:38.499395 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=3 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:45:38.558459 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=54 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:45:38.737043 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda18b44-dedf-437e-9021-480c7b4cc7a1 None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:45:38.747097 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda18b44-dedf-437e-9021-480c7b4cc7a1 None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:45:38.750735 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27af958-86bc-4cff-9f2d-edea7f7a8c4f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43ad98a250a144099ccd18378aedd4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:45:38.751333 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27af958-86bc-4cff-9f2d-edea7f7a8c4f service neutron] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:45:38.751613 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27af958-86bc-4cff-9f2d-edea7f7a8c4f service neutron] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:45:38.751889 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27af958-86bc-4cff-9f2d-edea7f7a8c4f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:45:38.753676 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-a27af958-86bc-4cff-9f2d-edea7f7a8c4f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:45:38.763254 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27af958-86bc-4cff-9f2d-edea7f7a8c4f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:45:38.764743 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27af958-86bc-4cff-9f2d-edea7f7a8c4f service neutron] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:45:38.773051 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1205/2394] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:45:38 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:45:40.769828 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa94a496-ecc8-4e93-af3e-ad04b5646122 None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:45:40.770324 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa94a496-ecc8-4e93-af3e-ad04b5646122 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:45:40.770457 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa94a496-ecc8-4e93-af3e-ad04b5646122 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:45:40.790600 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-fa94a496-ecc8-4e93-af3e-ad04b5646122 None None] MFA Rules not processed for user `40c0367417434c04948e9f2a03e880a9`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:45:40.792395 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-fa94a496-ecc8-4e93-af3e-ad04b5646122 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:45:40.801484 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1190/2395] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:45:40 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:45:48.769724 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=6 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:45:50.784716 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=3 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:45:56.472479 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27af958-86bc-4cff-9f2d-edea7f7a8c4f service neutron] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:45:56.474468 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-a27af958-86bc-4cff-9f2d-edea7f7a8c4f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:45:56.494724 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27af958-86bc-4cff-9f2d-edea7f7a8c4f service neutron] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:45:56.502530 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6143b09-5cc3-4d64-8a3a-47673886bc81 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '053b7ce6d6f649ed91a26e434dab805e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:45:56.504578 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6143b09-5cc3-4d64-8a3a-47673886bc81 service nova] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:45:56.505419 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6143b09-5cc3-4d64-8a3a-47673886bc81 service nova] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:45:56.506153 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6143b09-5cc3-4d64-8a3a-47673886bc81 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:45:56.508142 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-f6143b09-5cc3-4d64-8a3a-47673886bc81 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:45:56.521733 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6143b09-5cc3-4d64-8a3a-47673886bc81 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:45:56.524130 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6143b09-5cc3-4d64-8a3a-47673886bc81 service nova] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:45:56.539554 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1206/2396] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:45:56 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:45:57.998011 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89dec35-271d-424d-a241-edf4d8e49e6e None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:45:57.998501 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89dec35-271d-424d-a241-edf4d8e49e6e None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:45:57.998607 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89dec35-271d-424d-a241-edf4d8e49e6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:45:58.017128 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-f89dec35-271d-424d-a241-edf4d8e49e6e None None] MFA Rules not processed for user `af6710bfe62d4ba29dc48e2f7aba9ec5`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:45:58.018417 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-f89dec35-271d-424d-a241-edf4d8e49e6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:45:58.073400 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1191/2397] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:45:57 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:45:58.389044 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6143b09-5cc3-4d64-8a3a-47673886bc81 service nova] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:45:58.398046 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6143b09-5cc3-4d64-8a3a-47673886bc81 service nova] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:45:58.402714 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdaf9ed2-b49e-4dc5-92f6-259b49800cf3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43ad98a250a144099ccd18378aedd4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:45:58.403313 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaf9ed2-b49e-4dc5-92f6-259b49800cf3 service neutron] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:45:58.403589 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaf9ed2-b49e-4dc5-92f6-259b49800cf3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:45:58.403857 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaf9ed2-b49e-4dc5-92f6-259b49800cf3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:45:58.405614 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-fdaf9ed2-b49e-4dc5-92f6-259b49800cf3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:45:58.415513 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdaf9ed2-b49e-4dc5-92f6-259b49800cf3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:45:58.417038 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdaf9ed2-b49e-4dc5-92f6-259b49800cf3 service neutron] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:45:58.425590 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1207/2398] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:45:58 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:45:58.871548 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89dec35-271d-424d-a241-edf4d8e49e6e None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:45:58.873276 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-f89dec35-271d-424d-a241-edf4d8e49e6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:45:58.944855 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89dec35-271d-424d-a241-edf4d8e49e6e None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:45:58.950803 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2b861e-8e66-4cdd-a5ad-cb39fdb43814 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37def4721b554b9ebb7704a19642d15b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:45:58.951428 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2b861e-8e66-4cdd-a5ad-cb39fdb43814 service placement] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:45:58.952037 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2b861e-8e66-4cdd-a5ad-cb39fdb43814 service placement] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:45:58.952037 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2b861e-8e66-4cdd-a5ad-cb39fdb43814 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:45:58.953838 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-0c2b861e-8e66-4cdd-a5ad-cb39fdb43814 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:45:58.965518 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2b861e-8e66-4cdd-a5ad-cb39fdb43814 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:45:58.967109 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2b861e-8e66-4cdd-a5ad-cb39fdb43814 service placement] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:45:58.977014 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1192/2399] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:45:58 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:46:08.421860 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=12 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:46:08.972808 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=57 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:46:11.839270 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdaf9ed2-b49e-4dc5-92f6-259b49800cf3 service neutron] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:46:11.853623 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdaf9ed2-b49e-4dc5-92f6-259b49800cf3 service neutron] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:46:11.859192 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b83807-6c40-4ad1-afb1-8bc236a5519a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '053b7ce6d6f649ed91a26e434dab805e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:46:11.859802 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b83807-6c40-4ad1-afb1-8bc236a5519a service nova] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:46:11.860083 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b83807-6c40-4ad1-afb1-8bc236a5519a service nova] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:46:11.860408 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b83807-6c40-4ad1-afb1-8bc236a5519a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:46:11.862150 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-32b83807-6c40-4ad1-afb1-8bc236a5519a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:46:11.875578 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b83807-6c40-4ad1-afb1-8bc236a5519a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:46:11.877302 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b83807-6c40-4ad1-afb1-8bc236a5519a service nova] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:46:11.889096 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1208/2400] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:46:11 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:46:21.883551 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=6 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:46:22.969078 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2b861e-8e66-4cdd-a5ad-cb39fdb43814 service placement] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:46:22.981371 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2b861e-8e66-4cdd-a5ad-cb39fdb43814 service placement] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:46:22.986993 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2074ea-7266-494c-8275-bf82db040dce service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '053b7ce6d6f649ed91a26e434dab805e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:46:22.987660 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2074ea-7266-494c-8275-bf82db040dce service nova] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:46:22.987955 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2074ea-7266-494c-8275-bf82db040dce service nova] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:46:22.988233 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2074ea-7266-494c-8275-bf82db040dce service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:46:22.989996 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-cc2074ea-7266-494c-8275-bf82db040dce service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:46:23.001164 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2074ea-7266-494c-8275-bf82db040dce service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:46:23.002680 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2074ea-7266-494c-8275-bf82db040dce service nova] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:46:23.011674 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1193/2401] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:46:22 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:46:33.007467 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=6 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:46:34.505038 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af30006c-9e1b-488f-a559-7db2aecbba0b None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:46:34.505536 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af30006c-9e1b-488f-a559-7db2aecbba0b None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:46:34.505637 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af30006c-9e1b-488f-a559-7db2aecbba0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:46:34.525071 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-af30006c-9e1b-488f-a559-7db2aecbba0b None None] MFA Rules not processed for user `c36a9a0d92024734b1aa007a56c6328c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:46:34.526562 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-af30006c-9e1b-488f-a559-7db2aecbba0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:46:34.533850 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1209/2402] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:46:34 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:46:34.831675 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2074ea-7266-494c-8275-bf82db040dce service nova] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:46:34.841319 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2074ea-7266-494c-8275-bf82db040dce service nova] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:46:34.844869 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888b11d0-9f52-4bef-977a-f9f4140aff38 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43ad98a250a144099ccd18378aedd4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:46:34.845455 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888b11d0-9f52-4bef-977a-f9f4140aff38 service neutron] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:46:34.845730 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888b11d0-9f52-4bef-977a-f9f4140aff38 service neutron] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:46:34.846001 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888b11d0-9f52-4bef-977a-f9f4140aff38 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:46:34.847685 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-888b11d0-9f52-4bef-977a-f9f4140aff38 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:46:34.856868 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888b11d0-9f52-4bef-977a-f9f4140aff38 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:46:34.858273 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888b11d0-9f52-4bef-977a-f9f4140aff38 service neutron] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:46:34.865894 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1194/2403] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:46:34 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:46:44.520239 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=3 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:46:44.862460 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=6 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:46:45.955223 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af30006c-9e1b-488f-a559-7db2aecbba0b None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:46:45.968244 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af30006c-9e1b-488f-a559-7db2aecbba0b None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:46:45.973854 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f446a92b-2ba7-47a2-bc1e-f03b64db05b0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '053b7ce6d6f649ed91a26e434dab805e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:46:45.974458 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f446a92b-2ba7-47a2-bc1e-f03b64db05b0 service nova] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:46:45.974738 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f446a92b-2ba7-47a2-bc1e-f03b64db05b0 service nova] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:46:45.975014 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f446a92b-2ba7-47a2-bc1e-f03b64db05b0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:46:45.976766 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-f446a92b-2ba7-47a2-bc1e-f03b64db05b0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:46:46.033397 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f446a92b-2ba7-47a2-bc1e-f03b64db05b0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:46:46.035003 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f446a92b-2ba7-47a2-bc1e-f03b64db05b0 service nova] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:46:46.043820 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1210/2404] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:46:45 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:46:53.266187 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed4f12b-8043-4d4c-963a-471d442362a1 None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:46:53.266724 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed4f12b-8043-4d4c-963a-471d442362a1 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:46:53.266839 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed4f12b-8043-4d4c-963a-471d442362a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:46:53.287046 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-1ed4f12b-8043-4d4c-963a-471d442362a1 None None] MFA Rules not processed for user `892dad01ea684e6cb2d5cec659c6c4b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:46:53.288246 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-1ed4f12b-8043-4d4c-963a-471d442362a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:46:53.295078 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1195/2405] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:46:53 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:46:56.039731 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=28 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:47:03.282161 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=3 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:47:12.572986 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f446a92b-2ba7-47a2-bc1e-f03b64db05b0 service nova] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:47:12.582302 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f446a92b-2ba7-47a2-bc1e-f03b64db05b0 service nova] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:47:12.585951 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28713ace-b092-481d-9d7f-9ca5c4cba450 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43ad98a250a144099ccd18378aedd4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:47:12.586577 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28713ace-b092-481d-9d7f-9ca5c4cba450 service neutron] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:47:12.586857 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28713ace-b092-481d-9d7f-9ca5c4cba450 service neutron] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:47:12.587135 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28713ace-b092-481d-9d7f-9ca5c4cba450 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:47:12.588861 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-28713ace-b092-481d-9d7f-9ca5c4cba450 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:47:12.653727 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28713ace-b092-481d-9d7f-9ca5c4cba450 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:47:12.655174 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28713ace-b092-481d-9d7f-9ca5c4cba450 service neutron] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:47:12.665247 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1211/2406] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:47:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:47:15.825731 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed4f12b-8043-4d4c-963a-471d442362a1 None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:47:15.827186 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-1ed4f12b-8043-4d4c-963a-471d442362a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:47:15.842635 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed4f12b-8043-4d4c-963a-471d442362a1 None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:47:15.847661 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b33690b-5ea2-4255-a91d-878f3a740bb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:47:15.848281 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b33690b-5ea2-4255-a91d-878f3a740bb0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:47:15.848575 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b33690b-5ea2-4255-a91d-878f3a740bb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:47:15.848877 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b33690b-5ea2-4255-a91d-878f3a740bb0 admin admin] PATH_INFO: `/v3/users/40c0367417434c04948e9f2a03e880a9` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:47:15.849981 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b33690b-5ea2-4255-a91d-878f3a740bb0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=40c0367417434c04948e9f2a03e880a9)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:47:15.851889 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b33690b-5ea2-4255-a91d-878f3a740bb0 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:47:15.897594 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-0b33690b-5ea2-4255-a91d-878f3a740bb0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '40c0367417434c04948e9f2a03e880a9'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:47:15.898243 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-0b33690b-5ea2-4255-a91d-878f3a740bb0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '40c0367417434c04948e9f2a03e880a9'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:47:15.906159 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-0b33690b-5ea2-4255-a91d-878f3a740bb0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '40c0367417434c04948e9f2a03e880a9'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:47:15.906837 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-0b33690b-5ea2-4255-a91d-878f3a740bb0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '40c0367417434c04948e9f2a03e880a9'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:47:15.913996 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-0b33690b-5ea2-4255-a91d-878f3a740bb0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '40c0367417434c04948e9f2a03e880a9'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:47:15.931995 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1196/2407] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 17:47:15 2022] DELETE /identity/v3/users/40c0367417434c04948e9f2a03e880a9 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:47:16.101869 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28713ace-b092-481d-9d7f-9ca5c4cba450 service neutron] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:47:16.113411 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28713ace-b092-481d-9d7f-9ca5c4cba450 service neutron] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:47:16.171547 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f4e220-c652-4b90-97b2-d6f10a517318 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:47:16.172102 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f4e220-c652-4b90-97b2-d6f10a517318 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:47:16.172419 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f4e220-c652-4b90-97b2-d6f10a517318 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:47:16.172722 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f4e220-c652-4b90-97b2-d6f10a517318 admin admin] PATH_INFO: `/v3/projects/c6986df7cf5d4be09aac96f460903bcc` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:47:16.173838 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f4e220-c652-4b90-97b2-d6f10a517318 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c6986df7cf5d4be09aac96f460903bcc)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:47:16.175722 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f4e220-c652-4b90-97b2-d6f10a517318 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:47:16.220160 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-e1f4e220-c652-4b90-97b2-d6f10a517318 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c6986df7cf5d4be09aac96f460903bcc', 'request_id': 'req-e1f4e220-c652-4b90-97b2-d6f10a517318'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:47:16.226643 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-e1f4e220-c652-4b90-97b2-d6f10a517318 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c6986df7cf5d4be09aac96f460903bcc', 'request_id': 'req-e1f4e220-c652-4b90-97b2-d6f10a517318'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:47:16.232838 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-e1f4e220-c652-4b90-97b2-d6f10a517318 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c6986df7cf5d4be09aac96f460903bcc', 'request_id': 'req-e1f4e220-c652-4b90-97b2-d6f10a517318'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:47:16.242796 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-e1f4e220-c652-4b90-97b2-d6f10a517318 admin admin] The token cache is being invalidate because project c6986df7cf5d4be09aac96f460903bcc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81758) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:47:16.243207 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-e1f4e220-c652-4b90-97b2-d6f10a517318 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:47:16.243868 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-e1f4e220-c652-4b90-97b2-d6f10a517318 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:47:16.245387 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1212/2408] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 17:47:16 2022] DELETE /identity/v3/projects/c6986df7cf5d4be09aac96f460903bcc => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:47:16.250920 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61bdaca-4d53-44a0-8918-6968e85b9276 None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:47:16.251217 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61bdaca-4d53-44a0-8918-6968e85b9276 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:47:16.251475 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61bdaca-4d53-44a0-8918-6968e85b9276 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:47:16.271432 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-e61bdaca-4d53-44a0-8918-6968e85b9276 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:47:16.272653 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-e61bdaca-4d53-44a0-8918-6968e85b9276 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:47:16.330441 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1197/2409] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:47:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:47:16.338158 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63386de-b99c-450a-a4cc-dc9d9deb186e None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:47:16.338468 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63386de-b99c-450a-a4cc-dc9d9deb186e None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:47:16.338744 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63386de-b99c-450a-a4cc-dc9d9deb186e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:47:16.357588 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-f63386de-b99c-450a-a4cc-dc9d9deb186e None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:47:16.358776 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-f63386de-b99c-450a-a4cc-dc9d9deb186e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:47:16.366409 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1213/2410] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:47:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:47:16.370374 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61bdaca-4d53-44a0-8918-6968e85b9276 None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:47:16.380300 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61bdaca-4d53-44a0-8918-6968e85b9276 None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:47:16.384808 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfc1c63-5545-49ec-97b1-e4f178fa05de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:47:16.385391 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfc1c63-5545-49ec-97b1-e4f178fa05de admin admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:47:16.385667 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfc1c63-5545-49ec-97b1-e4f178fa05de admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:47:16.385936 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfc1c63-5545-49ec-97b1-e4f178fa05de admin admin] PATH_INFO: `/v3/domains` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:47:16.386569 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfc1c63-5545-49ec-97b1-e4f178fa05de admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:47:16.386869 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfc1c63-5545-49ec-97b1-e4f178fa05de admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:47:16.388442 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfc1c63-5545-49ec-97b1-e4f178fa05de admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:47:16.400560 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1198/2411] 10.222.0.45 () {58 vars in 1153 bytes} [Wed Aug 24 17:47:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:47:16.405080 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63386de-b99c-450a-a4cc-dc9d9deb186e None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:47:16.414087 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63386de-b99c-450a-a4cc-dc9d9deb186e None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:47:16.418600 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89234008-8717-4267-b6ce-b3ac7088b697 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:47:16.419206 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89234008-8717-4267-b6ce-b3ac7088b697 admin admin] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:47:16.419481 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89234008-8717-4267-b6ce-b3ac7088b697 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:47:16.419751 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89234008-8717-4267-b6ce-b3ac7088b697 admin admin] PATH_INFO: `/v3/projects` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:47:16.420362 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89234008-8717-4267-b6ce-b3ac7088b697 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:47:16.422722 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89234008-8717-4267-b6ce-b3ac7088b697 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:47:16.438733 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:47:16.451529 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1214/2412] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 17:47:16 2022] POST /identity/v3/projects => generated 375 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:47:16.456088 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfc1c63-5545-49ec-97b1-e4f178fa05de admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:47:16.465959 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfc1c63-5545-49ec-97b1-e4f178fa05de admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:47:16.522109 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f541628-dde2-48ba-8257-7cc5e2686718 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:47:16.522716 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f541628-dde2-48ba-8257-7cc5e2686718 admin admin] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:47:16.522997 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f541628-dde2-48ba-8257-7cc5e2686718 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:47:16.523269 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f541628-dde2-48ba-8257-7cc5e2686718 admin admin] PATH_INFO: `/v3/users` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:47:16.523893 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f541628-dde2-48ba-8257-7cc5e2686718 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:47:16.525886 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f541628-dde2-48ba-8257-7cc5e2686718 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:47:16.533865 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:47:16.548865 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1199/2413] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 17:47:16 2022] POST /identity/v3/users => generated 288 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:47:16.552876 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89234008-8717-4267-b6ce-b3ac7088b697 admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:47:16.563883 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89234008-8717-4267-b6ce-b3ac7088b697 admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:47:16.568432 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6aab83a-4352-4063-81b7-0fdf5f70657e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:47:16.569046 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6aab83a-4352-4063-81b7-0fdf5f70657e admin admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:47:16.569322 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6aab83a-4352-4063-81b7-0fdf5f70657e admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:47:16.569592 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6aab83a-4352-4063-81b7-0fdf5f70657e admin admin] PATH_INFO: `/v3/roles` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:47:16.570226 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6aab83a-4352-4063-81b7-0fdf5f70657e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:47:16.571804 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6aab83a-4352-4063-81b7-0fdf5f70657e admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:47:16.581436 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1215/2414] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:47:16 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:47:16.585436 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f541628-dde2-48ba-8257-7cc5e2686718 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:47:16.595248 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f541628-dde2-48ba-8257-7cc5e2686718 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:47:16.599799 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7595aca-2a00-4961-b63a-856d3ccddd4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:47:16.600386 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7595aca-2a00-4961-b63a-856d3ccddd4c admin admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:47:16.600696 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7595aca-2a00-4961-b63a-856d3ccddd4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:47:16.600973 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7595aca-2a00-4961-b63a-856d3ccddd4c admin admin] PATH_INFO: `/v3/roles` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:47:16.601605 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7595aca-2a00-4961-b63a-856d3ccddd4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:47:16.603247 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7595aca-2a00-4961-b63a-856d3ccddd4c admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:47:16.614054 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1200/2415] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:47:16 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:47:16.617895 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6aab83a-4352-4063-81b7-0fdf5f70657e admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:47:16.628004 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6aab83a-4352-4063-81b7-0fdf5f70657e admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:47:16.632471 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8be33a8-be86-4410-b0ab-4ca30a0bf6e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:47:16.633129 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8be33a8-be86-4410-b0ab-4ca30a0bf6e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:47:16.633403 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8be33a8-be86-4410-b0ab-4ca30a0bf6e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:47:16.633682 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8be33a8-be86-4410-b0ab-4ca30a0bf6e7 admin admin] PATH_INFO: `/v3/projects/75c39bcd24354fd5b79179c48abc07f1/users/58652b0d60ec4134be18dafeb7ad5cb0/roles/02476a5b82eb4b188c18a5d706a3ef68` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:47:16.647434 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8be33a8-be86-4410-b0ab-4ca30a0bf6e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=75c39bcd24354fd5b79179c48abc07f1, user_id=58652b0d60ec4134be18dafeb7ad5cb0, role_id=02476a5b82eb4b188c18a5d706a3ef68)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:47:16.650089 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8be33a8-be86-4410-b0ab-4ca30a0bf6e7 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:47:16.662950 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1216/2416] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 17:47:16 2022] PUT /identity/v3/projects/75c39bcd24354fd5b79179c48abc07f1/users/58652b0d60ec4134be18dafeb7ad5cb0/roles/02476a5b82eb4b188c18a5d706a3ef68 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:47:16.667821 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781061d8-676c-4d95-9597-be916a1d8276 None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:47:16.668106 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781061d8-676c-4d95-9597-be916a1d8276 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:47:16.668379 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781061d8-676c-4d95-9597-be916a1d8276 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:47:16.687817 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-781061d8-676c-4d95-9597-be916a1d8276 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:47:16.688980 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-781061d8-676c-4d95-9597-be916a1d8276 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:47:16.746781 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1201/2417] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:47:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:47:16.756822 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8be33a8-be86-4410-b0ab-4ca30a0bf6e7 admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:47:16.758485 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-f8be33a8-be86-4410-b0ab-4ca30a0bf6e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:47:16.814888 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8be33a8-be86-4410-b0ab-4ca30a0bf6e7 admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:47:16.818506 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5aad189-233d-4833-ae71-e41911b46f16 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43ad98a250a144099ccd18378aedd4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:47:16.819107 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5aad189-233d-4833-ae71-e41911b46f16 service neutron] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:47:16.819387 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5aad189-233d-4833-ae71-e41911b46f16 service neutron] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:47:16.819662 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5aad189-233d-4833-ae71-e41911b46f16 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:47:16.828352 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5aad189-233d-4833-ae71-e41911b46f16 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:47:16.829808 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5aad189-233d-4833-ae71-e41911b46f16 service neutron] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:47:16.839023 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1217/2418] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:47:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 17:47:20.658878 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d599e4-5b2e-43da-adcc-85f175c87025 None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:47:20.659468 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d599e4-5b2e-43da-adcc-85f175c87025 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:47:20.659468 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d599e4-5b2e-43da-adcc-85f175c87025 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:47:20.687866 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-c1d599e4-5b2e-43da-adcc-85f175c87025 None None] MFA Rules not processed for user `58652b0d60ec4134be18dafeb7ad5cb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:47:20.689105 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-c1d599e4-5b2e-43da-adcc-85f175c87025 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:47:20.783371 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1202/2419] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:47:20 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:47:20.804964 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5aad189-233d-4833-ae71-e41911b46f16 service neutron] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:47:20.807072 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-c5aad189-233d-4833-ae71-e41911b46f16 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:47:20.885443 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5aad189-233d-4833-ae71-e41911b46f16 service neutron] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:47:20.891597 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df763152-430c-4e6f-aca8-4f4dad015a1a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '053b7ce6d6f649ed91a26e434dab805e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:47:20.892215 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df763152-430c-4e6f-aca8-4f4dad015a1a service nova] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:47:20.892495 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df763152-430c-4e6f-aca8-4f4dad015a1a service nova] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:47:20.892802 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df763152-430c-4e6f-aca8-4f4dad015a1a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:47:20.901257 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df763152-430c-4e6f-aca8-4f4dad015a1a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:47:20.902833 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df763152-430c-4e6f-aca8-4f4dad015a1a service nova] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:47:20.912698 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1218/2420] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:47:20 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:47:22.262904 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63973b05-7879-4a48-8dab-8540e28471f5 None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:47:22.263497 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63973b05-7879-4a48-8dab-8540e28471f5 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:47:22.263613 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63973b05-7879-4a48-8dab-8540e28471f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:47:22.290138 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-63973b05-7879-4a48-8dab-8540e28471f5 None None] MFA Rules not processed for user `58652b0d60ec4134be18dafeb7ad5cb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:47:22.291357 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-63973b05-7879-4a48-8dab-8540e28471f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:47:22.298484 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1203/2421] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:47:22 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:47:30.908255 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=77,keystone:INSERT=1 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:47:32.283844 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:INSERT=3,keystone:SELECT=69 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:47:45.210534 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee28d5a7-b9a7-4503-8e28-f53d1990e58f None None] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:47:45.212016 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee28d5a7-b9a7-4503-8e28-f53d1990e58f None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:47:45.212300 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee28d5a7-b9a7-4503-8e28-f53d1990e58f None None] PATH_INFO: `/` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:47:45.213878 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1219/2422] 10.222.0.45 () {54 vars in 921 bytes} [Wed Aug 24 17:47:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:47:45.220228 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb8a8a5-1d1c-4d6b-bf37-5b7ea28ee5e5 None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:47:45.220752 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb8a8a5-1d1c-4d6b-bf37-5b7ea28ee5e5 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:47:45.221019 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb8a8a5-1d1c-4d6b-bf37-5b7ea28ee5e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:47:45.241557 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-5eb8a8a5-1d1c-4d6b-bf37-5b7ea28ee5e5 None None] MFA Rules not processed for user `43ad98a250a144099ccd18378aedd4b5`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:47:45.242775 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-5eb8a8a5-1d1c-4d6b-bf37-5b7ea28ee5e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:47:45.249895 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1204/2423] 10.222.0.45 () {58 vars in 1018 bytes} [Wed Aug 24 17:47:45 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:47:45.261670 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee28d5a7-b9a7-4503-8e28-f53d1990e58f None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:47:45.272841 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee28d5a7-b9a7-4503-8e28-f53d1990e58f None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:47:45.276888 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ba9fca-78ca-4d65-9e86-364505c8a72f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43ad98a250a144099ccd18378aedd4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:47:45.277604 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ba9fca-78ca-4d65-9e86-364505c8a72f service neutron] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:47:45.278070 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ba9fca-78ca-4d65-9e86-364505c8a72f service neutron] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:47:45.278354 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ba9fca-78ca-4d65-9e86-364505c8a72f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:47:45.286640 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ba9fca-78ca-4d65-9e86-364505c8a72f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:47:45.288067 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ba9fca-78ca-4d65-9e86-364505c8a72f service neutron] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:47:45.296839 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1220/2424] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:47:45 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:47:45.382395 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe4a8d1-070e-4e60-b028-f05bb0d14368 None None] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:47:45.382687 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe4a8d1-070e-4e60-b028-f05bb0d14368 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:47:45.382963 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe4a8d1-070e-4e60-b028-f05bb0d14368 None None] PATH_INFO: `/` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:47:45.384322 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1205/2425] 10.222.0.45 () {54 vars in 921 bytes} [Wed Aug 24 17:47:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:47:45.389323 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbdcfd1-7b19-4ff5-8b56-3e8a429819d1 None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:47:45.389614 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbdcfd1-7b19-4ff5-8b56-3e8a429819d1 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:47:45.389945 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbdcfd1-7b19-4ff5-8b56-3e8a429819d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:47:45.409912 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-9dbdcfd1-7b19-4ff5-8b56-3e8a429819d1 None None] MFA Rules not processed for user `053b7ce6d6f649ed91a26e434dab805e`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:47:45.411113 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-9dbdcfd1-7b19-4ff5-8b56-3e8a429819d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:47:45.423518 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1221/2426] 10.222.0.45 () {58 vars in 1018 bytes} [Wed Aug 24 17:47:45 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:47:45.434019 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe4a8d1-070e-4e60-b028-f05bb0d14368 None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:47:45.435706 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-8fe4a8d1-070e-4e60-b028-f05bb0d14368 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:47:45.449368 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe4a8d1-070e-4e60-b028-f05bb0d14368 None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:47:45.453429 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326cbf1b-5da4-4e70-b44c-efa3c9ac033c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43ad98a250a144099ccd18378aedd4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:47:45.454019 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326cbf1b-5da4-4e70-b44c-efa3c9ac033c service neutron] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:47:45.454297 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326cbf1b-5da4-4e70-b44c-efa3c9ac033c service neutron] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:47:45.454567 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326cbf1b-5da4-4e70-b44c-efa3c9ac033c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:47:45.464374 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326cbf1b-5da4-4e70-b44c-efa3c9ac033c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:47:45.465846 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326cbf1b-5da4-4e70-b44c-efa3c9ac033c service neutron] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:47:45.476281 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1206/2427] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:47:45 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:47:55.404333 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=9 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:47:55.471783 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=9 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:48:20.745937 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbdcfd1-7b19-4ff5-8b56-3e8a429819d1 None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:20.747319 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-9dbdcfd1-7b19-4ff5-8b56-3e8a429819d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:20.832922 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbdcfd1-7b19-4ff5-8b56-3e8a429819d1 None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:20.836533 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24fea72-f48f-4d22-a02e-567c9f5d3e7f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59be342541264942ad87f60ff4de6c8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:20.837182 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24fea72-f48f-4d22-a02e-567c9f5d3e7f service cinder] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:20.837463 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24fea72-f48f-4d22-a02e-567c9f5d3e7f service cinder] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:20.837744 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24fea72-f48f-4d22-a02e-567c9f5d3e7f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:20.839489 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-d24fea72-f48f-4d22-a02e-567c9f5d3e7f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:20.908873 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24fea72-f48f-4d22-a02e-567c9f5d3e7f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:20.910296 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24fea72-f48f-4d22-a02e-567c9f5d3e7f service cinder] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:20.919154 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1222/2428] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:48:20 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:48:30.914986 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=67 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:48:34.024612 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cffd07-e1fa-46c5-adf4-5aa32c2b799a None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:34.025095 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cffd07-e1fa-46c5-adf4-5aa32c2b799a None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:34.025192 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cffd07-e1fa-46c5-adf4-5aa32c2b799a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:34.044477 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-02cffd07-e1fa-46c5-adf4-5aa32c2b799a None None] MFA Rules not processed for user `af6710bfe62d4ba29dc48e2f7aba9ec5`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:48:34.045813 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-02cffd07-e1fa-46c5-adf4-5aa32c2b799a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:34.053153 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1207/2429] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:48:34 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:48:35.249759 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24fea72-f48f-4d22-a02e-567c9f5d3e7f service cinder] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:35.251933 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-d24fea72-f48f-4d22-a02e-567c9f5d3e7f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:35.348129 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24fea72-f48f-4d22-a02e-567c9f5d3e7f service cinder] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:35.353078 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b801edd-964a-4774-9698-122446be44ea service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37def4721b554b9ebb7704a19642d15b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:35.353667 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b801edd-964a-4774-9698-122446be44ea service placement] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:35.353943 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b801edd-964a-4774-9698-122446be44ea service placement] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:35.354216 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b801edd-964a-4774-9698-122446be44ea service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:35.355918 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-6b801edd-964a-4774-9698-122446be44ea service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:35.367310 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b801edd-964a-4774-9698-122446be44ea service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:35.369031 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b801edd-964a-4774-9698-122446be44ea service placement] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:35.379839 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1223/2430] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:48:35 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:48:41.233586 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff4ca72-228c-40c2-9778-14965eb868ad None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:41.234084 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff4ca72-228c-40c2-9778-14965eb868ad None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:41.234187 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff4ca72-228c-40c2-9778-14965eb868ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:41.254041 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-6ff4ca72-228c-40c2-9778-14965eb868ad None None] MFA Rules not processed for user `c36a9a0d92024734b1aa007a56c6328c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:48:41.255237 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-6ff4ca72-228c-40c2-9778-14965eb868ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:41.308648 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1208/2431] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:48:41 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:48:45.374572 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=44 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:48:51.302071 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=28 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:48:59.640386 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b801edd-964a-4774-9698-122446be44ea service placement] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:59.659115 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b801edd-964a-4774-9698-122446be44ea service placement] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:59.662661 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffc7410-d6b6-4da6-a784-8749c381da00 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43ad98a250a144099ccd18378aedd4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:59.663247 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffc7410-d6b6-4da6-a784-8749c381da00 service neutron] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:59.663524 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffc7410-d6b6-4da6-a784-8749c381da00 service neutron] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:59.663793 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffc7410-d6b6-4da6-a784-8749c381da00 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:59.665493 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-dffc7410-d6b6-4da6-a784-8749c381da00 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:59.733698 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffc7410-d6b6-4da6-a784-8749c381da00 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:59.735177 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffc7410-d6b6-4da6-a784-8749c381da00 service neutron] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:59.745265 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1224/2432] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:48:59 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:49:09.740914 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=34 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:49:18.646683 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ac4297-74e2-4e66-9701-813cef2ba3d2 None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:18.647197 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ac4297-74e2-4e66-9701-813cef2ba3d2 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:18.647312 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ac4297-74e2-4e66-9701-813cef2ba3d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:18.677094 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-e1ac4297-74e2-4e66-9701-813cef2ba3d2 None None] MFA Rules not processed for user `ea143d042ee94ae7a4544d90d18db7db`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:49:18.678515 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-e1ac4297-74e2-4e66-9701-813cef2ba3d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:18.739931 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1209/2433] 10.222.0.45 () {58 vars in 1009 bytes} [Wed Aug 24 17:49:18 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:49:18.786577 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffc7410-d6b6-4da6-a784-8749c381da00 service neutron] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:18.787994 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-dffc7410-d6b6-4da6-a784-8749c381da00 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:18.857481 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffc7410-d6b6-4da6-a784-8749c381da00 service neutron] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:18.862938 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db78e5d-e1e6-483b-b3c3-685f3e36b0ad service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90f0772b64c84faaa689e7272fa97898', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:18.863537 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db78e5d-e1e6-483b-b3c3-685f3e36b0ad service swift] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:18.863818 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db78e5d-e1e6-483b-b3c3-685f3e36b0ad service swift] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:18.864089 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db78e5d-e1e6-483b-b3c3-685f3e36b0ad service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:18.871809 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db78e5d-e1e6-483b-b3c3-685f3e36b0ad service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:18.873414 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db78e5d-e1e6-483b-b3c3-685f3e36b0ad service swift] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:18.882047 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1225/2434] 10.222.0.45 () {60 vars in 1384 bytes} [Wed Aug 24 17:49:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 24 17:49:28.733130 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=32 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:49:28.878627 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=32 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:49:37.990953 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf3fc2f-779b-40cf-a191-414211f75f34 None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:37.991450 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf3fc2f-779b-40cf-a191-414211f75f34 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:37.991547 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf3fc2f-779b-40cf-a191-414211f75f34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:38.015900 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-2bf3fc2f-779b-40cf-a191-414211f75f34 None None] MFA Rules not processed for user `ea143d042ee94ae7a4544d90d18db7db`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:49:38.017147 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-2bf3fc2f-779b-40cf-a191-414211f75f34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:38.024222 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1210/2435] 10.222.0.45 () {58 vars in 1009 bytes} [Wed Aug 24 17:49:37 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:49:38.036701 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db78e5d-e1e6-483b-b3c3-685f3e36b0ad service swift] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:38.046980 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db78e5d-e1e6-483b-b3c3-685f3e36b0ad service swift] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:38.052510 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db345726-884c-4dca-99df-6c1c1852d68b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90f0772b64c84faaa689e7272fa97898', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:38.053125 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db345726-884c-4dca-99df-6c1c1852d68b service swift] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:38.053401 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db345726-884c-4dca-99df-6c1c1852d68b service swift] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:38.053671 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db345726-884c-4dca-99df-6c1c1852d68b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:38.061580 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db345726-884c-4dca-99df-6c1c1852d68b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:38.063160 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db345726-884c-4dca-99df-6c1c1852d68b service swift] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:38.071204 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1226/2436] 10.222.0.45 () {60 vars in 1384 bytes} [Wed Aug 24 17:49:38 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 17:49:48.011020 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=3 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:49:48.067980 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=6 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:49:59.601343 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf3fc2f-779b-40cf-a191-414211f75f34 None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:59.610598 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf3fc2f-779b-40cf-a191-414211f75f34 None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:59.614590 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2f20fe-ce04-4872-ba51-61c2b31b8bc1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43ad98a250a144099ccd18378aedd4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:59.615181 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2f20fe-ce04-4872-ba51-61c2b31b8bc1 service neutron] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:59.615458 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2f20fe-ce04-4872-ba51-61c2b31b8bc1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:59.615731 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2f20fe-ce04-4872-ba51-61c2b31b8bc1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:59.617427 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-3a2f20fe-ce04-4872-ba51-61c2b31b8bc1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:59.673255 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2f20fe-ce04-4872-ba51-61c2b31b8bc1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:59.674674 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2f20fe-ce04-4872-ba51-61c2b31b8bc1 service neutron] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:59.683727 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1211/2437] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:49:59 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:50:02.528865 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a780a9d6-e5b7-4783-8b0b-d1c38c55fac7 None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:02.529336 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a780a9d6-e5b7-4783-8b0b-d1c38c55fac7 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:02.529429 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a780a9d6-e5b7-4783-8b0b-d1c38c55fac7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:02.548161 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-a780a9d6-e5b7-4783-8b0b-d1c38c55fac7 None None] MFA Rules not processed for user `ea143d042ee94ae7a4544d90d18db7db`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:50:02.549393 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-a780a9d6-e5b7-4783-8b0b-d1c38c55fac7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:02.556989 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1227/2438] 10.222.0.45 () {58 vars in 967 bytes} [Wed Aug 24 17:50:02 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:50:02.569578 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2f20fe-ce04-4872-ba51-61c2b31b8bc1 service neutron] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:02.581228 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2f20fe-ce04-4872-ba51-61c2b31b8bc1 service neutron] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:02.586395 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e558045d-f109-4b6c-be98-1eb0eec5b592 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90f0772b64c84faaa689e7272fa97898', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:02.587074 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e558045d-f109-4b6c-be98-1eb0eec5b592 service swift] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:02.587349 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e558045d-f109-4b6c-be98-1eb0eec5b592 service swift] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:02.587621 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e558045d-f109-4b6c-be98-1eb0eec5b592 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:02.595749 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e558045d-f109-4b6c-be98-1eb0eec5b592 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:02.597353 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e558045d-f109-4b6c-be98-1eb0eec5b592 service swift] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:02.605582 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1212/2439] 10.222.0.45 () {60 vars in 1384 bytes} [Wed Aug 24 17:50:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 17:50:06.912378 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a780a9d6-e5b7-4783-8b0b-d1c38c55fac7 None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:06.921703 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a780a9d6-e5b7-4783-8b0b-d1c38c55fac7 None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:06.925890 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89c42c7-5164-457e-ba94-c9f7d19d7f54 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43ad98a250a144099ccd18378aedd4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:06.926487 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89c42c7-5164-457e-ba94-c9f7d19d7f54 service neutron] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:06.926767 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89c42c7-5164-457e-ba94-c9f7d19d7f54 service neutron] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:06.927041 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89c42c7-5164-457e-ba94-c9f7d19d7f54 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:06.928886 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-c89c42c7-5164-457e-ba94-c9f7d19d7f54 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:06.961707 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89c42c7-5164-457e-ba94-c9f7d19d7f54 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:06.963148 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89c42c7-5164-457e-ba94-c9f7d19d7f54 service neutron] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:06.982008 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1228/2440] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:50:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 17:50:10.012815 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e558045d-f109-4b6c-be98-1eb0eec5b592 service swift] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:10.014294 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-e558045d-f109-4b6c-be98-1eb0eec5b592 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:10.031589 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e558045d-f109-4b6c-be98-1eb0eec5b592 service swift] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:10.040264 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca72543-d90e-47af-959a-f10e7b677c11 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37def4721b554b9ebb7704a19642d15b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:10.040973 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca72543-d90e-47af-959a-f10e7b677c11 service placement] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:10.041250 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca72543-d90e-47af-959a-f10e7b677c11 service placement] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:10.042138 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca72543-d90e-47af-959a-f10e7b677c11 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:10.043907 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-fca72543-d90e-47af-959a-f10e7b677c11 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:10.059181 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca72543-d90e-47af-959a-f10e7b677c11 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:10.061039 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca72543-d90e-47af-959a-f10e7b677c11 service placement] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:10.081201 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1213/2441] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:50:10 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:50:12.885255 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89c42c7-5164-457e-ba94-c9f7d19d7f54 service neutron] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:12.899309 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89c42c7-5164-457e-ba94-c9f7d19d7f54 service neutron] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:12.905643 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741a4384-5ee5-489c-a57e-c8208de88ef0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37def4721b554b9ebb7704a19642d15b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:12.906297 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741a4384-5ee5-489c-a57e-c8208de88ef0 service placement] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:12.906591 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741a4384-5ee5-489c-a57e-c8208de88ef0 service placement] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:12.906869 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741a4384-5ee5-489c-a57e-c8208de88ef0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:12.908761 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-741a4384-5ee5-489c-a57e-c8208de88ef0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:12.921971 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741a4384-5ee5-489c-a57e-c8208de88ef0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:12.923717 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741a4384-5ee5-489c-a57e-c8208de88ef0 service placement] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:12.937328 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1229/2442] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:50:12 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:50:12.960356 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca72543-d90e-47af-959a-f10e7b677c11 service placement] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:12.961951 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-fca72543-d90e-47af-959a-f10e7b677c11 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:12.976282 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca72543-d90e-47af-959a-f10e7b677c11 service placement] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:12.981204 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad76549-7966-4ee9-852b-46166de6fbd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:12.981901 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad76549-7966-4ee9-852b-46166de6fbd1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:12.982352 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad76549-7966-4ee9-852b-46166de6fbd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:12.982622 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad76549-7966-4ee9-852b-46166de6fbd1 admin admin] PATH_INFO: `/v3/users/8d38f974f55d4f27bafdfe06ac9dfebb` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:12.983904 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad76549-7966-4ee9-852b-46166de6fbd1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8d38f974f55d4f27bafdfe06ac9dfebb)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:12.985836 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad76549-7966-4ee9-852b-46166de6fbd1 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:13.032843 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-7ad76549-7966-4ee9-852b-46166de6fbd1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8d38f974f55d4f27bafdfe06ac9dfebb'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:13.033492 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-7ad76549-7966-4ee9-852b-46166de6fbd1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8d38f974f55d4f27bafdfe06ac9dfebb'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:13.041670 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-7ad76549-7966-4ee9-852b-46166de6fbd1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8d38f974f55d4f27bafdfe06ac9dfebb'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:13.042329 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-7ad76549-7966-4ee9-852b-46166de6fbd1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8d38f974f55d4f27bafdfe06ac9dfebb'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:13.049851 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-7ad76549-7966-4ee9-852b-46166de6fbd1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8d38f974f55d4f27bafdfe06ac9dfebb'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:13.067636 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1214/2443] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 17:50:12 2022] DELETE /identity/v3/users/8d38f974f55d4f27bafdfe06ac9dfebb => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:50:13.241441 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741a4384-5ee5-489c-a57e-c8208de88ef0 service placement] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:13.251987 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741a4384-5ee5-489c-a57e-c8208de88ef0 service placement] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:13.309994 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87b409c-9ffd-4aa3-9fd7-11f2f8cbfd68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:13.310616 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87b409c-9ffd-4aa3-9fd7-11f2f8cbfd68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:13.310896 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87b409c-9ffd-4aa3-9fd7-11f2f8cbfd68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:13.311170 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87b409c-9ffd-4aa3-9fd7-11f2f8cbfd68 admin admin] PATH_INFO: `/v3/projects/f45fcc8295e24580b04787b4040d3612` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:13.312265 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87b409c-9ffd-4aa3-9fd7-11f2f8cbfd68 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f45fcc8295e24580b04787b4040d3612)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:13.314192 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87b409c-9ffd-4aa3-9fd7-11f2f8cbfd68 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:13.359854 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-a87b409c-9ffd-4aa3-9fd7-11f2f8cbfd68 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f45fcc8295e24580b04787b4040d3612', 'request_id': 'req-a87b409c-9ffd-4aa3-9fd7-11f2f8cbfd68'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:13.367001 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-a87b409c-9ffd-4aa3-9fd7-11f2f8cbfd68 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f45fcc8295e24580b04787b4040d3612', 'request_id': 'req-a87b409c-9ffd-4aa3-9fd7-11f2f8cbfd68'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:13.372976 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-a87b409c-9ffd-4aa3-9fd7-11f2f8cbfd68 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f45fcc8295e24580b04787b4040d3612', 'request_id': 'req-a87b409c-9ffd-4aa3-9fd7-11f2f8cbfd68'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:13.382883 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-a87b409c-9ffd-4aa3-9fd7-11f2f8cbfd68 admin admin] The token cache is being invalidate because project f45fcc8295e24580b04787b4040d3612 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81758) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:50:13.383171 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-a87b409c-9ffd-4aa3-9fd7-11f2f8cbfd68 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:13.383786 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-a87b409c-9ffd-4aa3-9fd7-11f2f8cbfd68 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:13.385254 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1230/2444] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 17:50:13 2022] DELETE /identity/v3/projects/f45fcc8295e24580b04787b4040d3612 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:50:13.389215 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad76549-7966-4ee9-852b-46166de6fbd1 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:13.390603 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-7ad76549-7966-4ee9-852b-46166de6fbd1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:13.403917 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad76549-7966-4ee9-852b-46166de6fbd1 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:13.444795 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:50:13.461086 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40026bb9-7338-4da8-b2f9-8374d491a80a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:13.461712 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40026bb9-7338-4da8-b2f9-8374d491a80a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:13.461990 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40026bb9-7338-4da8-b2f9-8374d491a80a admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:13.462262 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40026bb9-7338-4da8-b2f9-8374d491a80a admin admin] PATH_INFO: `/v3/users/a1027018a8fd4b00bf27e2aed7d71204` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:13.463364 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40026bb9-7338-4da8-b2f9-8374d491a80a admin admin] RBAC: Authorizing `identity:delete_user(user_id=a1027018a8fd4b00bf27e2aed7d71204)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:13.465340 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40026bb9-7338-4da8-b2f9-8374d491a80a admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:13.508674 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-40026bb9-7338-4da8-b2f9-8374d491a80a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a1027018a8fd4b00bf27e2aed7d71204'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:13.509307 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-40026bb9-7338-4da8-b2f9-8374d491a80a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a1027018a8fd4b00bf27e2aed7d71204'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:13.516031 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-40026bb9-7338-4da8-b2f9-8374d491a80a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a1027018a8fd4b00bf27e2aed7d71204'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:13.516676 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-40026bb9-7338-4da8-b2f9-8374d491a80a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a1027018a8fd4b00bf27e2aed7d71204'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:13.524093 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-40026bb9-7338-4da8-b2f9-8374d491a80a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a1027018a8fd4b00bf27e2aed7d71204'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:13.542391 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1215/2445] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 17:50:13 2022] DELETE /identity/v3/users/a1027018a8fd4b00bf27e2aed7d71204 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:50:13.706098 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87b409c-9ffd-4aa3-9fd7-11f2f8cbfd68 admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:13.708075 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-a87b409c-9ffd-4aa3-9fd7-11f2f8cbfd68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:13.720778 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87b409c-9ffd-4aa3-9fd7-11f2f8cbfd68 admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:13.760073 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:50:13.791096 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5cc67d-43b3-4edb-9b9f-d8a84204643e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:13.791738 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5cc67d-43b3-4edb-9b9f-d8a84204643e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:13.792024 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5cc67d-43b3-4edb-9b9f-d8a84204643e admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:13.792303 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5cc67d-43b3-4edb-9b9f-d8a84204643e admin admin] PATH_INFO: `/v3/projects/61e6476063b748cbb69fb9b2bedfe34f` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:13.793455 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5cc67d-43b3-4edb-9b9f-d8a84204643e admin admin] RBAC: Authorizing `identity:delete_project(project_id=61e6476063b748cbb69fb9b2bedfe34f)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:13.795519 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5cc67d-43b3-4edb-9b9f-d8a84204643e admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:13.845515 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-3e5cc67d-43b3-4edb-9b9f-d8a84204643e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '61e6476063b748cbb69fb9b2bedfe34f', 'request_id': 'req-3e5cc67d-43b3-4edb-9b9f-d8a84204643e'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:13.853345 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-3e5cc67d-43b3-4edb-9b9f-d8a84204643e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '61e6476063b748cbb69fb9b2bedfe34f', 'request_id': 'req-3e5cc67d-43b3-4edb-9b9f-d8a84204643e'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:13.859684 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-3e5cc67d-43b3-4edb-9b9f-d8a84204643e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '61e6476063b748cbb69fb9b2bedfe34f', 'request_id': 'req-3e5cc67d-43b3-4edb-9b9f-d8a84204643e'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:13.869871 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-3e5cc67d-43b3-4edb-9b9f-d8a84204643e admin admin] The token cache is being invalidate because project 61e6476063b748cbb69fb9b2bedfe34f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81758) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:50:13.870162 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-3e5cc67d-43b3-4edb-9b9f-d8a84204643e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:13.870785 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-3e5cc67d-43b3-4edb-9b9f-d8a84204643e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:13.872256 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1231/2446] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 17:50:13 2022] DELETE /identity/v3/projects/61e6476063b748cbb69fb9b2bedfe34f => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:50:23.538788 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=33,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:50:23.866917 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=38,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:50:28.169564 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40026bb9-7338-4da8-b2f9-8374d491a80a admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:28.170944 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-40026bb9-7338-4da8-b2f9-8374d491a80a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:28.248241 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40026bb9-7338-4da8-b2f9-8374d491a80a admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:28.305324 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5d743a-1cd3-421e-bf79-6cf145ad495e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59be342541264942ad87f60ff4de6c8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:28.305698 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5d743a-1cd3-421e-bf79-6cf145ad495e service cinder] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:28.306451 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5d743a-1cd3-421e-bf79-6cf145ad495e service cinder] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:28.306737 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5d743a-1cd3-421e-bf79-6cf145ad495e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:28.308530 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-4b5d743a-1cd3-421e-bf79-6cf145ad495e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:28.392066 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5d743a-1cd3-421e-bf79-6cf145ad495e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:28.393545 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5d743a-1cd3-421e-bf79-6cf145ad495e service cinder] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:28.414969 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1216/2447] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:50:28 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:50:36.284603 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5cc67d-43b3-4edb-9b9f-d8a84204643e admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:36.286802 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-3e5cc67d-43b3-4edb-9b9f-d8a84204643e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:36.299276 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5cc67d-43b3-4edb-9b9f-d8a84204643e admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:36.302847 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe24be9a-7651-4155-af00-e6dff0a23586 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59be342541264942ad87f60ff4de6c8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:36.303547 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe24be9a-7651-4155-af00-e6dff0a23586 service cinder] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:36.303830 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe24be9a-7651-4155-af00-e6dff0a23586 service cinder] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:36.304105 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe24be9a-7651-4155-af00-e6dff0a23586 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:36.305770 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-fe24be9a-7651-4155-af00-e6dff0a23586 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:36.319083 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe24be9a-7651-4155-af00-e6dff0a23586 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:36.320515 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe24be9a-7651-4155-af00-e6dff0a23586 service cinder] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:36.332645 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1232/2448] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:50:36 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:50:38.412091 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=92 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:50:45.907507 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5d743a-1cd3-421e-bf79-6cf145ad495e service cinder] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:45.908931 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-4b5d743a-1cd3-421e-bf79-6cf145ad495e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:45.975263 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5d743a-1cd3-421e-bf79-6cf145ad495e service cinder] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:45.978879 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ee38b1-83a0-4cc4-a476-92910f96b457 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43ad98a250a144099ccd18378aedd4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:45.979418 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ee38b1-83a0-4cc4-a476-92910f96b457 service neutron] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:45.979697 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ee38b1-83a0-4cc4-a476-92910f96b457 service neutron] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:45.979972 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ee38b1-83a0-4cc4-a476-92910f96b457 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:45.981685 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-85ee38b1-83a0-4cc4-a476-92910f96b457 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:46.042672 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ee38b1-83a0-4cc4-a476-92910f96b457 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:46.044115 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ee38b1-83a0-4cc4-a476-92910f96b457 service neutron] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:46.057985 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1217/2449] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:50:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:50:46.326415 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=6 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:50:48.871417 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe24be9a-7651-4155-af00-e6dff0a23586 service cinder] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:48.872907 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-fe24be9a-7651-4155-af00-e6dff0a23586 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:48.887804 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe24be9a-7651-4155-af00-e6dff0a23586 service cinder] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:48.892498 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27be9c8-8515-4663-8721-78d3d34e41df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:48.893163 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27be9c8-8515-4663-8721-78d3d34e41df admin admin] REQUEST_METHOD: `DELETE` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:48.893475 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27be9c8-8515-4663-8721-78d3d34e41df admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:48.893750 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27be9c8-8515-4663-8721-78d3d34e41df admin admin] PATH_INFO: `/v3/users/c36a9a0d92024734b1aa007a56c6328c` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:48.913956 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27be9c8-8515-4663-8721-78d3d34e41df admin admin] RBAC: Authorizing `identity:delete_user(user_id=c36a9a0d92024734b1aa007a56c6328c)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:48.916253 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27be9c8-8515-4663-8721-78d3d34e41df admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:48.964621 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-c27be9c8-8515-4663-8721-78d3d34e41df admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c36a9a0d92024734b1aa007a56c6328c'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:48.965279 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-c27be9c8-8515-4663-8721-78d3d34e41df admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c36a9a0d92024734b1aa007a56c6328c'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:48.972349 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-c27be9c8-8515-4663-8721-78d3d34e41df admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c36a9a0d92024734b1aa007a56c6328c'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:48.973124 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-c27be9c8-8515-4663-8721-78d3d34e41df admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c36a9a0d92024734b1aa007a56c6328c'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:48.985217 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-c27be9c8-8515-4663-8721-78d3d34e41df admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c36a9a0d92024734b1aa007a56c6328c'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:49.001474 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1233/2450] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 17:50:48 2022] DELETE /identity/v3/users/c36a9a0d92024734b1aa007a56c6328c => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:50:49.168819 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ee38b1-83a0-4cc4-a476-92910f96b457 service neutron] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:49.179188 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ee38b1-83a0-4cc4-a476-92910f96b457 service neutron] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:49.234819 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae5338d-076e-4c2c-9573-7a3017f5c874 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:49.235446 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae5338d-076e-4c2c-9573-7a3017f5c874 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:49.235726 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae5338d-076e-4c2c-9573-7a3017f5c874 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:49.236001 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae5338d-076e-4c2c-9573-7a3017f5c874 admin admin] PATH_INFO: `/v3/projects/22d1c7b9f8a94e5abc2adbad97e5a479` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:49.246771 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae5338d-076e-4c2c-9573-7a3017f5c874 admin admin] RBAC: Authorizing `identity:delete_project(project_id=22d1c7b9f8a94e5abc2adbad97e5a479)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:49.248720 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae5338d-076e-4c2c-9573-7a3017f5c874 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:49.281576 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:50:49.295169 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-eae5338d-076e-4c2c-9573-7a3017f5c874 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '22d1c7b9f8a94e5abc2adbad97e5a479', 'request_id': 'req-eae5338d-076e-4c2c-9573-7a3017f5c874'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:49.301376 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-eae5338d-076e-4c2c-9573-7a3017f5c874 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '22d1c7b9f8a94e5abc2adbad97e5a479', 'request_id': 'req-eae5338d-076e-4c2c-9573-7a3017f5c874'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:49.307967 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-eae5338d-076e-4c2c-9573-7a3017f5c874 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '22d1c7b9f8a94e5abc2adbad97e5a479', 'request_id': 'req-eae5338d-076e-4c2c-9573-7a3017f5c874'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:49.320234 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-eae5338d-076e-4c2c-9573-7a3017f5c874 admin admin] The token cache is being invalidate because project 22d1c7b9f8a94e5abc2adbad97e5a479 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81759) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:50:49.320518 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-eae5338d-076e-4c2c-9573-7a3017f5c874 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:49.321168 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-eae5338d-076e-4c2c-9573-7a3017f5c874 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:49.322618 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1218/2451] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 17:50:49 2022] DELETE /identity/v3/projects/22d1c7b9f8a94e5abc2adbad97e5a479 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:50:49.327513 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3b9fa6-f5ab-4f36-ad59-d37f7196a261 None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:49.327803 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3b9fa6-f5ab-4f36-ad59-d37f7196a261 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:49.328081 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3b9fa6-f5ab-4f36-ad59-d37f7196a261 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:49.383764 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-ea3b9fa6-f5ab-4f36-ad59-d37f7196a261 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:50:49.385001 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-ea3b9fa6-f5ab-4f36-ad59-d37f7196a261 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:49.440930 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1234/2452] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:50:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:50:49.449327 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e7c5f8-ccd6-48e3-9ca4-edb25f8d68c3 None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:49.449607 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e7c5f8-ccd6-48e3-9ca4-edb25f8d68c3 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:49.449878 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e7c5f8-ccd6-48e3-9ca4-edb25f8d68c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:49.470331 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-a6e7c5f8-ccd6-48e3-9ca4-edb25f8d68c3 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:50:49.471511 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-a6e7c5f8-ccd6-48e3-9ca4-edb25f8d68c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:49.479810 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1219/2453] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:50:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:50:49.483849 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3b9fa6-f5ab-4f36-ad59-d37f7196a261 None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:49.493165 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3b9fa6-f5ab-4f36-ad59-d37f7196a261 None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:49.497601 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5031e685-7284-4654-b02f-5153c67b5cca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:49.498188 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5031e685-7284-4654-b02f-5153c67b5cca admin admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:49.498462 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5031e685-7284-4654-b02f-5153c67b5cca admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:49.498732 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5031e685-7284-4654-b02f-5153c67b5cca admin admin] PATH_INFO: `/v3/domains` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:49.499369 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5031e685-7284-4654-b02f-5153c67b5cca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:50:49.499683 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5031e685-7284-4654-b02f-5153c67b5cca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:49.501289 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5031e685-7284-4654-b02f-5153c67b5cca admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:49.512468 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1235/2454] 10.222.0.45 () {58 vars in 1153 bytes} [Wed Aug 24 17:50:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:50:49.516605 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e7c5f8-ccd6-48e3-9ca4-edb25f8d68c3 None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:49.526189 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e7c5f8-ccd6-48e3-9ca4-edb25f8d68c3 None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:49.530783 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12307f79-0c6a-4a8b-8993-fde23e3e4626 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:49.531369 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12307f79-0c6a-4a8b-8993-fde23e3e4626 admin admin] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:49.531646 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12307f79-0c6a-4a8b-8993-fde23e3e4626 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:49.531912 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12307f79-0c6a-4a8b-8993-fde23e3e4626 admin admin] PATH_INFO: `/v3/projects` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:49.532534 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12307f79-0c6a-4a8b-8993-fde23e3e4626 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:49.534445 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12307f79-0c6a-4a8b-8993-fde23e3e4626 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:49.570108 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1220/2455] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 17:50:49 2022] POST /identity/v3/projects => generated 365 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:50:49.574245 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5031e685-7284-4654-b02f-5153c67b5cca admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:49.584731 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5031e685-7284-4654-b02f-5153c67b5cca admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:49.596437 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:50:49.640200 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bec817-6734-49d6-83b4-6a49a8c058c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:49.640828 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bec817-6734-49d6-83b4-6a49a8c058c6 admin admin] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:49.641106 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bec817-6734-49d6-83b4-6a49a8c058c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:49.641375 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bec817-6734-49d6-83b4-6a49a8c058c6 admin admin] PATH_INFO: `/v3/users` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:49.641993 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bec817-6734-49d6-83b4-6a49a8c058c6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:49.643879 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bec817-6734-49d6-83b4-6a49a8c058c6 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:49.665070 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1236/2456] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 17:50:49 2022] POST /identity/v3/users => generated 283 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:50:49.669151 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12307f79-0c6a-4a8b-8993-fde23e3e4626 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:49.679456 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12307f79-0c6a-4a8b-8993-fde23e3e4626 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:49.684166 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0610f94-0c9f-4988-a322-96af423081ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:49.684797 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0610f94-0c9f-4988-a322-96af423081ba admin admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:49.685077 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0610f94-0c9f-4988-a322-96af423081ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:49.685347 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0610f94-0c9f-4988-a322-96af423081ba admin admin] PATH_INFO: `/v3/roles` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:49.685986 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0610f94-0c9f-4988-a322-96af423081ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:49.687567 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0610f94-0c9f-4988-a322-96af423081ba admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:49.697367 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1221/2457] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:50:49 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:50:49.701176 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bec817-6734-49d6-83b4-6a49a8c058c6 admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:49.711453 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bec817-6734-49d6-83b4-6a49a8c058c6 admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:49.716011 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9392bdf3-e0a6-4940-8adc-a8700e5e534a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:49.716636 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9392bdf3-e0a6-4940-8adc-a8700e5e534a admin admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:49.716918 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9392bdf3-e0a6-4940-8adc-a8700e5e534a admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:49.717192 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9392bdf3-e0a6-4940-8adc-a8700e5e534a admin admin] PATH_INFO: `/v3/roles` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:49.717826 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9392bdf3-e0a6-4940-8adc-a8700e5e534a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:49.719422 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9392bdf3-e0a6-4940-8adc-a8700e5e534a admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:49.730350 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1237/2458] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:50:49 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:50:49.734429 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0610f94-0c9f-4988-a322-96af423081ba admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:49.744801 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0610f94-0c9f-4988-a322-96af423081ba admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:49.749702 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0aecc22-32c7-4920-835c-12855e5dd77f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:49.750347 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0aecc22-32c7-4920-835c-12855e5dd77f admin admin] REQUEST_METHOD: `PUT` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:49.750711 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0aecc22-32c7-4920-835c-12855e5dd77f admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:49.750889 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0aecc22-32c7-4920-835c-12855e5dd77f admin admin] PATH_INFO: `/v3/projects/5d94b666bf7943b6b09b8a28d00adb64/users/81b7764f80ca40158b38cca2cf2bdd97/roles/02476a5b82eb4b188c18a5d706a3ef68` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:49.766445 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0aecc22-32c7-4920-835c-12855e5dd77f admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d94b666bf7943b6b09b8a28d00adb64, user_id=81b7764f80ca40158b38cca2cf2bdd97, role_id=02476a5b82eb4b188c18a5d706a3ef68)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:49.769650 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0aecc22-32c7-4920-835c-12855e5dd77f admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:49.788991 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1222/2459] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 17:50:49 2022] PUT /identity/v3/projects/5d94b666bf7943b6b09b8a28d00adb64/users/81b7764f80ca40158b38cca2cf2bdd97/roles/02476a5b82eb4b188c18a5d706a3ef68 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:50:49.799192 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f0e725-5d05-4d2f-85b5-040569e1838e None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:49.799692 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f0e725-5d05-4d2f-85b5-040569e1838e None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:49.800415 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f0e725-5d05-4d2f-85b5-040569e1838e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:49.827613 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-27f0e725-5d05-4d2f-85b5-040569e1838e None None] MFA Rules not processed for user `81b7764f80ca40158b38cca2cf2bdd97`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:50:49.828883 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-27f0e725-5d05-4d2f-85b5-040569e1838e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:49.893216 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1238/2460] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:50:49 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:50:49.898557 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0aecc22-32c7-4920-835c-12855e5dd77f admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:49.959521 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0aecc22-32c7-4920-835c-12855e5dd77f admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:49.964136 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5def9d20-782b-4185-8720-6d7778a5e6e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:49.964767 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5def9d20-782b-4185-8720-6d7778a5e6e1 admin admin] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:49.965049 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5def9d20-782b-4185-8720-6d7778a5e6e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:49.965325 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5def9d20-782b-4185-8720-6d7778a5e6e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:49.965939 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5def9d20-782b-4185-8720-6d7778a5e6e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:49.967843 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5def9d20-782b-4185-8720-6d7778a5e6e1 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:49.993592 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1223/2461] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 17:50:49 2022] POST /identity/v3/projects => generated 365 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:50:49.997711 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f0e725-5d05-4d2f-85b5-040569e1838e None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:50.008684 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f0e725-5d05-4d2f-85b5-040569e1838e None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:50.063384 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c26e9f-9912-46e2-a08b-d87dbf312a83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:50.063974 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c26e9f-9912-46e2-a08b-d87dbf312a83 admin admin] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:50.064251 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c26e9f-9912-46e2-a08b-d87dbf312a83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:50.064520 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c26e9f-9912-46e2-a08b-d87dbf312a83 admin admin] PATH_INFO: `/v3/users` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:50.065170 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c26e9f-9912-46e2-a08b-d87dbf312a83 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:50.067562 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c26e9f-9912-46e2-a08b-d87dbf312a83 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:50.087842 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1239/2462] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 17:50:49 2022] POST /identity/v3/users => generated 289 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:50:50.091679 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5def9d20-782b-4185-8720-6d7778a5e6e1 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:50.101931 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5def9d20-782b-4185-8720-6d7778a5e6e1 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:50.106480 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1836fcc0-ac38-4ea9-9cc1-559ab1329307 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:50.107073 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1836fcc0-ac38-4ea9-9cc1-559ab1329307 admin admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:50.107355 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1836fcc0-ac38-4ea9-9cc1-559ab1329307 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:50.107630 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1836fcc0-ac38-4ea9-9cc1-559ab1329307 admin admin] PATH_INFO: `/v3/roles` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:50.108265 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1836fcc0-ac38-4ea9-9cc1-559ab1329307 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:50.109899 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1836fcc0-ac38-4ea9-9cc1-559ab1329307 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:50.120225 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1224/2463] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:50:50 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 17:50:50.124305 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c26e9f-9912-46e2-a08b-d87dbf312a83 admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:50.134861 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c26e9f-9912-46e2-a08b-d87dbf312a83 admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:50.139572 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a25985-d84b-43cb-9b25-3f555baab185 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:50.140226 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a25985-d84b-43cb-9b25-3f555baab185 admin admin] REQUEST_METHOD: `PUT` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:50.140507 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a25985-d84b-43cb-9b25-3f555baab185 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:50.140818 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a25985-d84b-43cb-9b25-3f555baab185 admin admin] PATH_INFO: `/v3/projects/5c441790be4a451482cad6b0ed5aede3/users/58b98fca9553420cb0ddd19f502171dc/roles/e411ad9eb96e4e8d85b59b5b700ada7b` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:50.153249 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:50:50.156511 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a25985-d84b-43cb-9b25-3f555baab185 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c441790be4a451482cad6b0ed5aede3, user_id=58b98fca9553420cb0ddd19f502171dc, role_id=e411ad9eb96e4e8d85b59b5b700ada7b)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:50.159696 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a25985-d84b-43cb-9b25-3f555baab185 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:50.173579 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1240/2464] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 17:50:50 2022] PUT /identity/v3/projects/5c441790be4a451482cad6b0ed5aede3/users/58b98fca9553420cb0ddd19f502171dc/roles/e411ad9eb96e4e8d85b59b5b700ada7b => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:50:50.177583 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1836fcc0-ac38-4ea9-9cc1-559ab1329307 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:50.187602 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1836fcc0-ac38-4ea9-9cc1-559ab1329307 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:50.205566 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:50:50.245638 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3099a032-454b-49ee-99ce-5ceb848be6cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:50.246163 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3099a032-454b-49ee-99ce-5ceb848be6cd admin admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:50.246475 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3099a032-454b-49ee-99ce-5ceb848be6cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:50.246733 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3099a032-454b-49ee-99ce-5ceb848be6cd admin admin] PATH_INFO: `/v3/roles` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:50.247369 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3099a032-454b-49ee-99ce-5ceb848be6cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:50.249128 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3099a032-454b-49ee-99ce-5ceb848be6cd admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:50.259353 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1225/2465] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:50:50 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:50:50.263485 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a25985-d84b-43cb-9b25-3f555baab185 admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:50.273973 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a25985-d84b-43cb-9b25-3f555baab185 admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:50.278564 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b5f9be-c206-4b70-adde-da5081aa932b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:50.279291 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b5f9be-c206-4b70-adde-da5081aa932b admin admin] REQUEST_METHOD: `PUT` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:50.279583 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b5f9be-c206-4b70-adde-da5081aa932b admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:50.279842 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b5f9be-c206-4b70-adde-da5081aa932b admin admin] PATH_INFO: `/v3/projects/5c441790be4a451482cad6b0ed5aede3/users/58b98fca9553420cb0ddd19f502171dc/roles/e411ad9eb96e4e8d85b59b5b700ada7b` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:50.281734 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b5f9be-c206-4b70-adde-da5081aa932b admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c441790be4a451482cad6b0ed5aede3, user_id=58b98fca9553420cb0ddd19f502171dc, role_id=e411ad9eb96e4e8d85b59b5b700ada7b)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:50.284348 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b5f9be-c206-4b70-adde-da5081aa932b admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:50.295237 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1241/2466] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 17:50:50 2022] PUT /identity/v3/projects/5c441790be4a451482cad6b0ed5aede3/users/58b98fca9553420cb0ddd19f502171dc/roles/e411ad9eb96e4e8d85b59b5b700ada7b => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:50:50.306613 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76e1ca5-4acc-4c7d-a1ac-6adf3f949dee None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:50.306899 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76e1ca5-4acc-4c7d-a1ac-6adf3f949dee None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:50.307172 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76e1ca5-4acc-4c7d-a1ac-6adf3f949dee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:50.334004 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-b76e1ca5-4acc-4c7d-a1ac-6adf3f949dee None None] MFA Rules not processed for user `58b98fca9553420cb0ddd19f502171dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:50:50.335223 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-b76e1ca5-4acc-4c7d-a1ac-6adf3f949dee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:50.401019 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1226/2467] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:50:50 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:50:50.412914 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b5f9be-c206-4b70-adde-da5081aa932b admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:50.414613 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-62b5f9be-c206-4b70-adde-da5081aa932b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:50.470745 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b5f9be-c206-4b70-adde-da5081aa932b admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:50.474993 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7170451e-dde0-4aea-b022-9a4a24875300 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59be342541264942ad87f60ff4de6c8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:50.475589 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7170451e-dde0-4aea-b022-9a4a24875300 service cinder] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:50.475870 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7170451e-dde0-4aea-b022-9a4a24875300 service cinder] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:50.476147 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7170451e-dde0-4aea-b022-9a4a24875300 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:50.484955 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7170451e-dde0-4aea-b022-9a4a24875300 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:50.486378 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7170451e-dde0-4aea-b022-9a4a24875300 service cinder] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:50.495517 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1242/2468] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:50:50 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:50:50.569289 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76e1ca5-4acc-4c7d-a1ac-6adf3f949dee None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:50.579029 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76e1ca5-4acc-4c7d-a1ac-6adf3f949dee None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:50.583263 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0794aed7-5dcf-4da5-bfbd-ff5b6244e9f3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59be342541264942ad87f60ff4de6c8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:50.583861 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0794aed7-5dcf-4da5-bfbd-ff5b6244e9f3 service cinder] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:50.584140 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0794aed7-5dcf-4da5-bfbd-ff5b6244e9f3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:50.584414 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0794aed7-5dcf-4da5-bfbd-ff5b6244e9f3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:50.638744 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0794aed7-5dcf-4da5-bfbd-ff5b6244e9f3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:50.640180 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0794aed7-5dcf-4da5-bfbd-ff5b6244e9f3 service cinder] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:50.649478 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1227/2469] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:50:50 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:51:00.491394 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=34,keystone:INSERT=2 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:51:00.646031 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=81 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:51:01.929922 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7170451e-dde0-4aea-b022-9a4a24875300 service cinder] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:01.993114 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7170451e-dde0-4aea-b022-9a4a24875300 service cinder] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:01.997542 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e53cbe-3544-4739-8168-b690106ced6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:01.998163 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e53cbe-3544-4739-8168-b690106ced6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:01.998451 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e53cbe-3544-4739-8168-b690106ced6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:01.998726 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e53cbe-3544-4739-8168-b690106ced6e admin admin] PATH_INFO: `/v3/users/81b7764f80ca40158b38cca2cf2bdd97` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:01.999858 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e53cbe-3544-4739-8168-b690106ced6e admin admin] RBAC: Authorizing `identity:delete_user(user_id=81b7764f80ca40158b38cca2cf2bdd97)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:02.001799 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e53cbe-3544-4739-8168-b690106ced6e admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:02.045482 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-45e53cbe-3544-4739-8168-b690106ced6e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '81b7764f80ca40158b38cca2cf2bdd97'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:02.046114 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-45e53cbe-3544-4739-8168-b690106ced6e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '81b7764f80ca40158b38cca2cf2bdd97'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:02.053363 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-45e53cbe-3544-4739-8168-b690106ced6e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '81b7764f80ca40158b38cca2cf2bdd97'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:02.053979 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-45e53cbe-3544-4739-8168-b690106ced6e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '81b7764f80ca40158b38cca2cf2bdd97'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:02.065416 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-45e53cbe-3544-4739-8168-b690106ced6e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '81b7764f80ca40158b38cca2cf2bdd97'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:02.077512 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1243/2470] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 17:51:01 2022] DELETE /identity/v3/users/81b7764f80ca40158b38cca2cf2bdd97 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:51:02.082015 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b803fd-40fe-4c70-8ab6-c67af45f2d13 None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:02.082491 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b803fd-40fe-4c70-8ab6-c67af45f2d13 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:02.082714 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b803fd-40fe-4c70-8ab6-c67af45f2d13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:02.101613 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-d2b803fd-40fe-4c70-8ab6-c67af45f2d13 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:51:02.102850 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-d2b803fd-40fe-4c70-8ab6-c67af45f2d13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:02.160693 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1228/2471] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:51:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:51:02.171304 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e53cbe-3544-4739-8168-b690106ced6e admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:02.172744 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-45e53cbe-3544-4739-8168-b690106ced6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:02.228220 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e53cbe-3544-4739-8168-b690106ced6e admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:02.231869 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc04811d-a334-4264-8c61-1e84fb40065b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43ad98a250a144099ccd18378aedd4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:02.232463 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc04811d-a334-4264-8c61-1e84fb40065b service neutron] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:02.232779 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc04811d-a334-4264-8c61-1e84fb40065b service neutron] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:02.233055 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc04811d-a334-4264-8c61-1e84fb40065b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:02.243441 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc04811d-a334-4264-8c61-1e84fb40065b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:02.245158 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc04811d-a334-4264-8c61-1e84fb40065b service neutron] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:02.254928 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1244/2472] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:51:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:51:02.518350 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b803fd-40fe-4c70-8ab6-c67af45f2d13 None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:02.519760 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-d2b803fd-40fe-4c70-8ab6-c67af45f2d13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:02.531411 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b803fd-40fe-4c70-8ab6-c67af45f2d13 None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:02.536932 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8beb365d-6357-49cc-b867-f6843aa8276b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:02.538198 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8beb365d-6357-49cc-b867-f6843aa8276b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:02.538320 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8beb365d-6357-49cc-b867-f6843aa8276b admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:02.538518 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8beb365d-6357-49cc-b867-f6843aa8276b admin admin] PATH_INFO: `/v3/projects/5d94b666bf7943b6b09b8a28d00adb64` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:02.539674 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8beb365d-6357-49cc-b867-f6843aa8276b admin admin] RBAC: Authorizing `identity:delete_project(project_id=5d94b666bf7943b6b09b8a28d00adb64)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:02.541835 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8beb365d-6357-49cc-b867-f6843aa8276b admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:02.591480 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-8beb365d-6357-49cc-b867-f6843aa8276b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5d94b666bf7943b6b09b8a28d00adb64', 'request_id': 'req-8beb365d-6357-49cc-b867-f6843aa8276b'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:02.596884 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-8beb365d-6357-49cc-b867-f6843aa8276b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5d94b666bf7943b6b09b8a28d00adb64', 'request_id': 'req-8beb365d-6357-49cc-b867-f6843aa8276b'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:02.603511 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-8beb365d-6357-49cc-b867-f6843aa8276b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5d94b666bf7943b6b09b8a28d00adb64', 'request_id': 'req-8beb365d-6357-49cc-b867-f6843aa8276b'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:02.617377 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-8beb365d-6357-49cc-b867-f6843aa8276b admin admin] The token cache is being invalidate because project 5d94b666bf7943b6b09b8a28d00adb64 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81759) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:51:02.617669 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-8beb365d-6357-49cc-b867-f6843aa8276b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:02.618308 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-8beb365d-6357-49cc-b867-f6843aa8276b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:02.619797 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1229/2473] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 17:51:02 2022] DELETE /identity/v3/projects/5d94b666bf7943b6b09b8a28d00adb64 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:51:02.623739 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc04811d-a334-4264-8c61-1e84fb40065b service neutron] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:02.633281 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc04811d-a334-4264-8c61-1e84fb40065b service neutron] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:02.650501 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:51:02.692310 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5152745-bd43-4700-a1fb-14b3079c0623 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:02.692969 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5152745-bd43-4700-a1fb-14b3079c0623 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:02.693263 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5152745-bd43-4700-a1fb-14b3079c0623 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:02.693518 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5152745-bd43-4700-a1fb-14b3079c0623 admin admin] PATH_INFO: `/v3/users/58b98fca9553420cb0ddd19f502171dc` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:02.694727 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5152745-bd43-4700-a1fb-14b3079c0623 admin admin] RBAC: Authorizing `identity:delete_user(user_id=58b98fca9553420cb0ddd19f502171dc)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:02.696666 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5152745-bd43-4700-a1fb-14b3079c0623 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:02.740588 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-c5152745-bd43-4700-a1fb-14b3079c0623 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '58b98fca9553420cb0ddd19f502171dc'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:02.741229 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-c5152745-bd43-4700-a1fb-14b3079c0623 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '58b98fca9553420cb0ddd19f502171dc'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:02.748047 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-c5152745-bd43-4700-a1fb-14b3079c0623 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '58b98fca9553420cb0ddd19f502171dc'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:02.748706 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-c5152745-bd43-4700-a1fb-14b3079c0623 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '58b98fca9553420cb0ddd19f502171dc'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:02.760250 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-c5152745-bd43-4700-a1fb-14b3079c0623 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '58b98fca9553420cb0ddd19f502171dc'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:02.772646 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1245/2474] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 17:51:02 2022] DELETE /identity/v3/users/58b98fca9553420cb0ddd19f502171dc => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:51:03.038226 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8beb365d-6357-49cc-b867-f6843aa8276b admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:03.039695 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-8beb365d-6357-49cc-b867-f6843aa8276b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:03.052963 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8beb365d-6357-49cc-b867-f6843aa8276b admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:03.111873 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43acd772-877e-4101-8294-0e611b1a4f62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:03.112542 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43acd772-877e-4101-8294-0e611b1a4f62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:03.112808 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43acd772-877e-4101-8294-0e611b1a4f62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:03.113087 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43acd772-877e-4101-8294-0e611b1a4f62 admin admin] PATH_INFO: `/v3/projects/5c441790be4a451482cad6b0ed5aede3` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:03.114196 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43acd772-877e-4101-8294-0e611b1a4f62 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5c441790be4a451482cad6b0ed5aede3)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:03.116130 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43acd772-877e-4101-8294-0e611b1a4f62 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:03.144510 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:51:03.162941 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-43acd772-877e-4101-8294-0e611b1a4f62 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5c441790be4a451482cad6b0ed5aede3', 'request_id': 'req-43acd772-877e-4101-8294-0e611b1a4f62'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:03.168479 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-43acd772-877e-4101-8294-0e611b1a4f62 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5c441790be4a451482cad6b0ed5aede3', 'request_id': 'req-43acd772-877e-4101-8294-0e611b1a4f62'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:03.174582 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-43acd772-877e-4101-8294-0e611b1a4f62 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5c441790be4a451482cad6b0ed5aede3', 'request_id': 'req-43acd772-877e-4101-8294-0e611b1a4f62'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:03.186621 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-43acd772-877e-4101-8294-0e611b1a4f62 admin admin] The token cache is being invalidate because project 5c441790be4a451482cad6b0ed5aede3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81759) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:51:03.186910 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-43acd772-877e-4101-8294-0e611b1a4f62 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:03.187529 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-43acd772-877e-4101-8294-0e611b1a4f62 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:03.189017 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1230/2475] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 17:51:03 2022] DELETE /identity/v3/projects/5c441790be4a451482cad6b0ed5aede3 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:51:03.193927 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2ef6e9-d0bf-4322-b4e9-5518a96c5ac7 None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:03.194386 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2ef6e9-d0bf-4322-b4e9-5518a96c5ac7 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:03.194481 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2ef6e9-d0bf-4322-b4e9-5518a96c5ac7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:03.214105 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-aa2ef6e9-d0bf-4322-b4e9-5518a96c5ac7 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:51:03.215296 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-aa2ef6e9-d0bf-4322-b4e9-5518a96c5ac7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:03.271414 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1246/2476] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:51:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:51:03.279111 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324647d2-2438-4249-a062-18718f90f6af None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:03.279393 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324647d2-2438-4249-a062-18718f90f6af None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:03.279668 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324647d2-2438-4249-a062-18718f90f6af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:03.300810 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-324647d2-2438-4249-a062-18718f90f6af None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:51:03.301926 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-324647d2-2438-4249-a062-18718f90f6af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:03.309654 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1231/2477] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:51:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:51:03.313896 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2ef6e9-d0bf-4322-b4e9-5518a96c5ac7 None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:03.323921 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2ef6e9-d0bf-4322-b4e9-5518a96c5ac7 None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:03.328414 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f819435-ec22-4e77-ac2d-42a12aa9c824 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:03.329035 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f819435-ec22-4e77-ac2d-42a12aa9c824 admin admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:03.329315 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f819435-ec22-4e77-ac2d-42a12aa9c824 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:03.329587 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f819435-ec22-4e77-ac2d-42a12aa9c824 admin admin] PATH_INFO: `/v3/domains` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:03.330229 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f819435-ec22-4e77-ac2d-42a12aa9c824 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:51:03.330532 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f819435-ec22-4e77-ac2d-42a12aa9c824 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:03.332117 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f819435-ec22-4e77-ac2d-42a12aa9c824 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:03.344614 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1247/2478] 10.222.0.45 () {58 vars in 1153 bytes} [Wed Aug 24 17:51:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:51:03.348644 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324647d2-2438-4249-a062-18718f90f6af None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:03.357613 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324647d2-2438-4249-a062-18718f90f6af None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:03.362244 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e6bb67-38fd-4803-87cc-e5896e8dcbbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:03.362827 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e6bb67-38fd-4803-87cc-e5896e8dcbbc admin admin] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:03.363100 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e6bb67-38fd-4803-87cc-e5896e8dcbbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:03.363368 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e6bb67-38fd-4803-87cc-e5896e8dcbbc admin admin] PATH_INFO: `/v3/projects` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:03.363975 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e6bb67-38fd-4803-87cc-e5896e8dcbbc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:03.365883 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e6bb67-38fd-4803-87cc-e5896e8dcbbc admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:03.395568 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1232/2479] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 17:51:03 2022] POST /identity/v3/projects => generated 369 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:51:03.397281 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f819435-ec22-4e77-ac2d-42a12aa9c824 admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:03.407073 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f819435-ec22-4e77-ac2d-42a12aa9c824 admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:03.432121 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:51:03.465257 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67753473-7fc9-42c0-b8c6-18bc0e44c34a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:03.465852 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67753473-7fc9-42c0-b8c6-18bc0e44c34a admin admin] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:03.466129 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67753473-7fc9-42c0-b8c6-18bc0e44c34a admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:03.466399 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67753473-7fc9-42c0-b8c6-18bc0e44c34a admin admin] PATH_INFO: `/v3/users` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:03.467005 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67753473-7fc9-42c0-b8c6-18bc0e44c34a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:03.468970 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67753473-7fc9-42c0-b8c6-18bc0e44c34a admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:03.488527 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1248/2480] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 17:51:03 2022] POST /identity/v3/users => generated 285 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:51:03.492701 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e6bb67-38fd-4803-87cc-e5896e8dcbbc admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:03.503085 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e6bb67-38fd-4803-87cc-e5896e8dcbbc admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:03.508265 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64993c6b-0f65-41d8-b709-cd6ed9445101 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:03.508903 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64993c6b-0f65-41d8-b709-cd6ed9445101 admin admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:03.509185 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64993c6b-0f65-41d8-b709-cd6ed9445101 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:03.509484 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64993c6b-0f65-41d8-b709-cd6ed9445101 admin admin] PATH_INFO: `/v3/roles` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:03.510122 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64993c6b-0f65-41d8-b709-cd6ed9445101 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:03.511720 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64993c6b-0f65-41d8-b709-cd6ed9445101 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:03.522345 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1233/2481] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:51:03 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:51:03.526159 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67753473-7fc9-42c0-b8c6-18bc0e44c34a admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:03.536036 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67753473-7fc9-42c0-b8c6-18bc0e44c34a admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:03.540923 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c17e344-2b4d-4531-a9a1-287c98d759f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:03.541510 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c17e344-2b4d-4531-a9a1-287c98d759f1 admin admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:03.541791 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c17e344-2b4d-4531-a9a1-287c98d759f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:03.542065 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c17e344-2b4d-4531-a9a1-287c98d759f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:03.542704 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c17e344-2b4d-4531-a9a1-287c98d759f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:03.544290 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c17e344-2b4d-4531-a9a1-287c98d759f1 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:03.554673 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1249/2482] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:51:03 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:51:03.558383 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64993c6b-0f65-41d8-b709-cd6ed9445101 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:03.568471 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64993c6b-0f65-41d8-b709-cd6ed9445101 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:03.573043 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eeaeaca-e25b-476e-bb35-f5ab1dc73c50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:03.573681 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eeaeaca-e25b-476e-bb35-f5ab1dc73c50 admin admin] REQUEST_METHOD: `PUT` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:03.573957 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eeaeaca-e25b-476e-bb35-f5ab1dc73c50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:03.574228 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eeaeaca-e25b-476e-bb35-f5ab1dc73c50 admin admin] PATH_INFO: `/v3/projects/c749b8ba0bf24a41ae4bd6e75e5fae23/users/00c43b90986a41e5a7b469d0ab707ccd/roles/02476a5b82eb4b188c18a5d706a3ef68` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:03.589208 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eeaeaca-e25b-476e-bb35-f5ab1dc73c50 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c749b8ba0bf24a41ae4bd6e75e5fae23, user_id=00c43b90986a41e5a7b469d0ab707ccd, role_id=02476a5b82eb4b188c18a5d706a3ef68)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:03.591847 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eeaeaca-e25b-476e-bb35-f5ab1dc73c50 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:03.604008 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1234/2483] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 17:51:03 2022] PUT /identity/v3/projects/c749b8ba0bf24a41ae4bd6e75e5fae23/users/00c43b90986a41e5a7b469d0ab707ccd/roles/02476a5b82eb4b188c18a5d706a3ef68 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:51:03.614322 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b2ee27-f32c-4600-ba1f-bd4c3a343949 None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:03.614606 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b2ee27-f32c-4600-ba1f-bd4c3a343949 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:03.614880 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b2ee27-f32c-4600-ba1f-bd4c3a343949 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:03.641712 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-f4b2ee27-f32c-4600-ba1f-bd4c3a343949 None None] MFA Rules not processed for user `00c43b90986a41e5a7b469d0ab707ccd`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:51:03.642906 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-f4b2ee27-f32c-4600-ba1f-bd4c3a343949 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:03.705948 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1250/2484] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:51:03 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:51:03.710149 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eeaeaca-e25b-476e-bb35-f5ab1dc73c50 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:03.771503 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eeaeaca-e25b-476e-bb35-f5ab1dc73c50 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:03.776354 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f844b9ab-f846-478f-b6b3-cc85cf26ee72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:03.776993 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f844b9ab-f846-478f-b6b3-cc85cf26ee72 admin admin] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:03.777272 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f844b9ab-f846-478f-b6b3-cc85cf26ee72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:03.777548 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f844b9ab-f846-478f-b6b3-cc85cf26ee72 admin admin] PATH_INFO: `/v3/projects` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:03.778180 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f844b9ab-f846-478f-b6b3-cc85cf26ee72 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:03.780104 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f844b9ab-f846-478f-b6b3-cc85cf26ee72 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:03.807106 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1235/2485] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 17:51:03 2022] POST /identity/v3/projects => generated 369 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:51:03.811105 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b2ee27-f32c-4600-ba1f-bd4c3a343949 None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:03.822282 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b2ee27-f32c-4600-ba1f-bd4c3a343949 None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:03.876385 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537d4cd9-8dd9-4aa6-9a1a-b2cc7fb53203 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:03.877035 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537d4cd9-8dd9-4aa6-9a1a-b2cc7fb53203 admin admin] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:03.877335 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537d4cd9-8dd9-4aa6-9a1a-b2cc7fb53203 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:03.877629 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537d4cd9-8dd9-4aa6-9a1a-b2cc7fb53203 admin admin] PATH_INFO: `/v3/users` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:03.878250 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537d4cd9-8dd9-4aa6-9a1a-b2cc7fb53203 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:03.880766 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537d4cd9-8dd9-4aa6-9a1a-b2cc7fb53203 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:03.901652 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1251/2486] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 17:51:03 2022] POST /identity/v3/users => generated 291 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:51:03.905991 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f844b9ab-f846-478f-b6b3-cc85cf26ee72 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:03.916493 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f844b9ab-f846-478f-b6b3-cc85cf26ee72 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:03.921059 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc18fa73-6caa-4b23-b637-4f0cc22ee3a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:03.921653 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc18fa73-6caa-4b23-b637-4f0cc22ee3a7 admin admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:03.921929 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc18fa73-6caa-4b23-b637-4f0cc22ee3a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:03.922199 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc18fa73-6caa-4b23-b637-4f0cc22ee3a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:03.922831 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc18fa73-6caa-4b23-b637-4f0cc22ee3a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:03.924410 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc18fa73-6caa-4b23-b637-4f0cc22ee3a7 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:03.935359 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1236/2487] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:51:03 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:51:03.939280 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537d4cd9-8dd9-4aa6-9a1a-b2cc7fb53203 admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:03.949678 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537d4cd9-8dd9-4aa6-9a1a-b2cc7fb53203 admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:03.954232 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cacdf83-9826-400a-b82d-f012385478c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:03.954901 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cacdf83-9826-400a-b82d-f012385478c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:03.955179 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cacdf83-9826-400a-b82d-f012385478c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:03.955451 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cacdf83-9826-400a-b82d-f012385478c2 admin admin] PATH_INFO: `/v3/projects/4c151730bf0b4e3aa87965b9fe30c589/users/b0713f9b8a41428f8a4e76abbac45c27/roles/e411ad9eb96e4e8d85b59b5b700ada7b` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:03.970844 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cacdf83-9826-400a-b82d-f012385478c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c151730bf0b4e3aa87965b9fe30c589, user_id=b0713f9b8a41428f8a4e76abbac45c27, role_id=e411ad9eb96e4e8d85b59b5b700ada7b)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:03.973534 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cacdf83-9826-400a-b82d-f012385478c2 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:03.989773 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1252/2488] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 17:51:03 2022] PUT /identity/v3/projects/4c151730bf0b4e3aa87965b9fe30c589/users/b0713f9b8a41428f8a4e76abbac45c27/roles/e411ad9eb96e4e8d85b59b5b700ada7b => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:51:03.993796 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc18fa73-6caa-4b23-b637-4f0cc22ee3a7 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:04.003760 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc18fa73-6caa-4b23-b637-4f0cc22ee3a7 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:04.023779 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:51:04.063306 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5090773-49a2-428b-b739-f3eb35e58252 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:04.063836 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5090773-49a2-428b-b739-f3eb35e58252 admin admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:04.064115 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5090773-49a2-428b-b739-f3eb35e58252 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:04.064391 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5090773-49a2-428b-b739-f3eb35e58252 admin admin] PATH_INFO: `/v3/roles` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:04.065068 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5090773-49a2-428b-b739-f3eb35e58252 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:04.066680 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5090773-49a2-428b-b739-f3eb35e58252 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:04.076445 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1237/2489] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:51:03 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:51:04.080693 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cacdf83-9826-400a-b82d-f012385478c2 admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:04.086216 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:51:04.091361 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cacdf83-9826-400a-b82d-f012385478c2 admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:04.097128 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce3b2b5-bb9b-4d97-9dde-939ff370ece3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:04.097818 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce3b2b5-bb9b-4d97-9dde-939ff370ece3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:04.098099 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce3b2b5-bb9b-4d97-9dde-939ff370ece3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:04.098376 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce3b2b5-bb9b-4d97-9dde-939ff370ece3 admin admin] PATH_INFO: `/v3/projects/4c151730bf0b4e3aa87965b9fe30c589/users/b0713f9b8a41428f8a4e76abbac45c27/roles/e411ad9eb96e4e8d85b59b5b700ada7b` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:04.100984 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce3b2b5-bb9b-4d97-9dde-939ff370ece3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c151730bf0b4e3aa87965b9fe30c589, user_id=b0713f9b8a41428f8a4e76abbac45c27, role_id=e411ad9eb96e4e8d85b59b5b700ada7b)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:04.103853 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce3b2b5-bb9b-4d97-9dde-939ff370ece3 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:04.115442 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1253/2490] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 17:51:04 2022] PUT /identity/v3/projects/4c151730bf0b4e3aa87965b9fe30c589/users/b0713f9b8a41428f8a4e76abbac45c27/roles/e411ad9eb96e4e8d85b59b5b700ada7b => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:51:04.127174 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a683f21e-936d-4d5f-8387-cb38b1d9b06b None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:04.127657 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a683f21e-936d-4d5f-8387-cb38b1d9b06b None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:04.127865 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a683f21e-936d-4d5f-8387-cb38b1d9b06b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:04.155714 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-a683f21e-936d-4d5f-8387-cb38b1d9b06b None None] MFA Rules not processed for user `b0713f9b8a41428f8a4e76abbac45c27`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:51:04.156959 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-a683f21e-936d-4d5f-8387-cb38b1d9b06b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:04.225543 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1238/2491] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:51:04 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:51:04.237355 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce3b2b5-bb9b-4d97-9dde-939ff370ece3 admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:04.239101 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-bce3b2b5-bb9b-4d97-9dde-939ff370ece3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:04.298991 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce3b2b5-bb9b-4d97-9dde-939ff370ece3 admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:04.303377 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4d1e80-2b5a-479e-9f1e-0249722060df service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59be342541264942ad87f60ff4de6c8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:04.303993 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4d1e80-2b5a-479e-9f1e-0249722060df service cinder] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:04.304269 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4d1e80-2b5a-479e-9f1e-0249722060df service cinder] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:04.304541 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4d1e80-2b5a-479e-9f1e-0249722060df service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:04.314593 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4d1e80-2b5a-479e-9f1e-0249722060df service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:04.315978 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4d1e80-2b5a-479e-9f1e-0249722060df service cinder] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:04.326421 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1254/2492] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:51:04 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:51:04.375652 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a683f21e-936d-4d5f-8387-cb38b1d9b06b None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:04.377112 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-a683f21e-936d-4d5f-8387-cb38b1d9b06b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:04.389817 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a683f21e-936d-4d5f-8387-cb38b1d9b06b None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:04.393807 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7937b179-b9fd-410d-9e45-c55cb8583285 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59be342541264942ad87f60ff4de6c8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:04.394684 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7937b179-b9fd-410d-9e45-c55cb8583285 service cinder] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:04.394964 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7937b179-b9fd-410d-9e45-c55cb8583285 service cinder] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:04.395247 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7937b179-b9fd-410d-9e45-c55cb8583285 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:04.455279 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7937b179-b9fd-410d-9e45-c55cb8583285 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:04.456768 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7937b179-b9fd-410d-9e45-c55cb8583285 service cinder] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:04.466458 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1239/2493] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:51:04 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:51:05.129275 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4d1e80-2b5a-479e-9f1e-0249722060df service cinder] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:05.131603 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-dc4d1e80-2b5a-479e-9f1e-0249722060df service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:05.215591 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4d1e80-2b5a-479e-9f1e-0249722060df service cinder] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:05.221482 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92dbe544-127f-422d-8068-c50a78027a70 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '053b7ce6d6f649ed91a26e434dab805e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:05.222154 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dbe544-127f-422d-8068-c50a78027a70 service nova] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:05.222525 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dbe544-127f-422d-8068-c50a78027a70 service nova] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:05.222795 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dbe544-127f-422d-8068-c50a78027a70 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:05.225143 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-92dbe544-127f-422d-8068-c50a78027a70 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:05.238235 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92dbe544-127f-422d-8068-c50a78027a70 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:05.240383 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92dbe544-127f-422d-8068-c50a78027a70 service nova] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:05.254044 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1255/2494] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:51:05 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:51:06.483335 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7937b179-b9fd-410d-9e45-c55cb8583285 service cinder] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:06.484969 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-7937b179-b9fd-410d-9e45-c55cb8583285 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:06.532161 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:51:06.543264 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7937b179-b9fd-410d-9e45-c55cb8583285 service cinder] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:06.546838 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc237c7-14b4-422d-951e-18744bf653db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43ad98a250a144099ccd18378aedd4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:06.547445 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc237c7-14b4-422d-951e-18744bf653db service neutron] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:06.547720 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc237c7-14b4-422d-951e-18744bf653db service neutron] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:06.547993 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc237c7-14b4-422d-951e-18744bf653db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:06.549749 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-4fc237c7-14b4-422d-951e-18744bf653db service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:06.559524 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc237c7-14b4-422d-951e-18744bf653db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:06.560996 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc237c7-14b4-422d-951e-18744bf653db service neutron] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:06.570445 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1240/2495] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:51:06 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:51:07.808870 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92dbe544-127f-422d-8068-c50a78027a70 service nova] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:07.873209 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92dbe544-127f-422d-8068-c50a78027a70 service nova] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:07.877736 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf8ad28-16ff-4fe5-b367-551a9b028728 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:07.878384 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf8ad28-16ff-4fe5-b367-551a9b028728 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:07.878665 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf8ad28-16ff-4fe5-b367-551a9b028728 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:07.878942 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf8ad28-16ff-4fe5-b367-551a9b028728 admin admin] PATH_INFO: `/v3/users/00c43b90986a41e5a7b469d0ab707ccd` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:07.881245 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf8ad28-16ff-4fe5-b367-551a9b028728 admin admin] RBAC: Authorizing `identity:delete_user(user_id=00c43b90986a41e5a7b469d0ab707ccd)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:07.883213 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf8ad28-16ff-4fe5-b367-551a9b028728 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:07.906233 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=97,keystone:INSERT=1,keystone:DELETE=2 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:51:07.935673 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-edf8ad28-16ff-4fe5-b367-551a9b028728 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '00c43b90986a41e5a7b469d0ab707ccd'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:07.936334 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-edf8ad28-16ff-4fe5-b367-551a9b028728 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '00c43b90986a41e5a7b469d0ab707ccd'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:07.944329 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-edf8ad28-16ff-4fe5-b367-551a9b028728 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '00c43b90986a41e5a7b469d0ab707ccd'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:07.945187 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-edf8ad28-16ff-4fe5-b367-551a9b028728 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '00c43b90986a41e5a7b469d0ab707ccd'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:07.958167 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-edf8ad28-16ff-4fe5-b367-551a9b028728 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '00c43b90986a41e5a7b469d0ab707ccd'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:07.972802 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1256/2496] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 17:51:07 2022] DELETE /identity/v3/users/00c43b90986a41e5a7b469d0ab707ccd => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:51:07.977531 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbed722-5212-42be-88e7-8fda27812405 None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:07.978065 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbed722-5212-42be-88e7-8fda27812405 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:07.978163 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbed722-5212-42be-88e7-8fda27812405 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:07.998866 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-dfbed722-5212-42be-88e7-8fda27812405 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:51:08.000095 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-dfbed722-5212-42be-88e7-8fda27812405 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:08.060402 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1241/2497] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:51:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:51:08.070682 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf8ad28-16ff-4fe5-b367-551a9b028728 admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:08.072208 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-edf8ad28-16ff-4fe5-b367-551a9b028728 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:08.130798 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf8ad28-16ff-4fe5-b367-551a9b028728 admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:08.134605 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faee8c23-8ec2-41ce-9903-4ba89837df5e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43ad98a250a144099ccd18378aedd4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:08.135285 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faee8c23-8ec2-41ce-9903-4ba89837df5e service neutron] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:08.135634 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faee8c23-8ec2-41ce-9903-4ba89837df5e service neutron] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:08.135974 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faee8c23-8ec2-41ce-9903-4ba89837df5e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:08.148767 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faee8c23-8ec2-41ce-9903-4ba89837df5e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:08.150149 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faee8c23-8ec2-41ce-9903-4ba89837df5e service neutron] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:08.163826 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1257/2498] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:51:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:51:08.252941 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbed722-5212-42be-88e7-8fda27812405 None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:08.254776 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-dfbed722-5212-42be-88e7-8fda27812405 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:08.357638 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbed722-5212-42be-88e7-8fda27812405 None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:08.363111 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710df5ce-f971-4f13-a2b0-a75206be053f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '053b7ce6d6f649ed91a26e434dab805e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:08.363714 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710df5ce-f971-4f13-a2b0-a75206be053f service nova] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:08.363993 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710df5ce-f971-4f13-a2b0-a75206be053f service nova] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:08.364266 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710df5ce-f971-4f13-a2b0-a75206be053f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:08.366547 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-710df5ce-f971-4f13-a2b0-a75206be053f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:08.378795 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710df5ce-f971-4f13-a2b0-a75206be053f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:08.381058 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710df5ce-f971-4f13-a2b0-a75206be053f service nova] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:08.391557 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1242/2499] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:51:08 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:51:08.449821 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faee8c23-8ec2-41ce-9903-4ba89837df5e service neutron] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:08.451255 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-faee8c23-8ec2-41ce-9903-4ba89837df5e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:08.463850 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faee8c23-8ec2-41ce-9903-4ba89837df5e service neutron] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:08.468418 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b35a03-a582-4e72-9a21-07bea1e072f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:08.469072 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b35a03-a582-4e72-9a21-07bea1e072f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:08.469348 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b35a03-a582-4e72-9a21-07bea1e072f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:08.469621 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b35a03-a582-4e72-9a21-07bea1e072f7 admin admin] PATH_INFO: `/v3/projects/c749b8ba0bf24a41ae4bd6e75e5fae23` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:08.470782 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b35a03-a582-4e72-9a21-07bea1e072f7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c749b8ba0bf24a41ae4bd6e75e5fae23)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:08.472714 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b35a03-a582-4e72-9a21-07bea1e072f7 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:08.523475 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-f7b35a03-a582-4e72-9a21-07bea1e072f7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c749b8ba0bf24a41ae4bd6e75e5fae23', 'request_id': 'req-f7b35a03-a582-4e72-9a21-07bea1e072f7'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:08.533843 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-f7b35a03-a582-4e72-9a21-07bea1e072f7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c749b8ba0bf24a41ae4bd6e75e5fae23', 'request_id': 'req-f7b35a03-a582-4e72-9a21-07bea1e072f7'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:08.541529 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-f7b35a03-a582-4e72-9a21-07bea1e072f7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c749b8ba0bf24a41ae4bd6e75e5fae23', 'request_id': 'req-f7b35a03-a582-4e72-9a21-07bea1e072f7'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:08.554858 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-f7b35a03-a582-4e72-9a21-07bea1e072f7 admin admin] The token cache is being invalidate because project c749b8ba0bf24a41ae4bd6e75e5fae23 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81758) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:51:08.555854 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-f7b35a03-a582-4e72-9a21-07bea1e072f7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:08.556144 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-f7b35a03-a582-4e72-9a21-07bea1e072f7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:08.558460 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1258/2500] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 17:51:08 2022] DELETE /identity/v3/projects/c749b8ba0bf24a41ae4bd6e75e5fae23 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:51:08.562126 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710df5ce-f971-4f13-a2b0-a75206be053f service nova] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:08.574141 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710df5ce-f971-4f13-a2b0-a75206be053f service nova] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:08.643628 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b5550c-05fe-4f5c-b8c9-2d2fa872ec70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:08.644276 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b5550c-05fe-4f5c-b8c9-2d2fa872ec70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:08.644576 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b5550c-05fe-4f5c-b8c9-2d2fa872ec70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:08.644920 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b5550c-05fe-4f5c-b8c9-2d2fa872ec70 admin admin] PATH_INFO: `/v3/users/b0713f9b8a41428f8a4e76abbac45c27` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:08.646154 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b5550c-05fe-4f5c-b8c9-2d2fa872ec70 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b0713f9b8a41428f8a4e76abbac45c27)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:08.648329 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b5550c-05fe-4f5c-b8c9-2d2fa872ec70 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:08.663608 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:51:08.701204 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-d0b5550c-05fe-4f5c-b8c9-2d2fa872ec70 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b0713f9b8a41428f8a4e76abbac45c27'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:08.701860 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-d0b5550c-05fe-4f5c-b8c9-2d2fa872ec70 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b0713f9b8a41428f8a4e76abbac45c27'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:08.709219 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-d0b5550c-05fe-4f5c-b8c9-2d2fa872ec70 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b0713f9b8a41428f8a4e76abbac45c27'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:08.709842 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-d0b5550c-05fe-4f5c-b8c9-2d2fa872ec70 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b0713f9b8a41428f8a4e76abbac45c27'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:08.717015 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-d0b5550c-05fe-4f5c-b8c9-2d2fa872ec70 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b0713f9b8a41428f8a4e76abbac45c27'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:08.734507 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1243/2501] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 17:51:08 2022] DELETE /identity/v3/users/b0713f9b8a41428f8a4e76abbac45c27 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:51:09.006447 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b35a03-a582-4e72-9a21-07bea1e072f7 admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:09.008085 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-f7b35a03-a582-4e72-9a21-07bea1e072f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:09.022802 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b35a03-a582-4e72-9a21-07bea1e072f7 admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:09.042453 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:51:09.082081 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3e1202-b55a-46a2-95df-a7fdbb328e25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:09.082713 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3e1202-b55a-46a2-95df-a7fdbb328e25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:09.082989 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3e1202-b55a-46a2-95df-a7fdbb328e25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:09.083355 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3e1202-b55a-46a2-95df-a7fdbb328e25 admin admin] PATH_INFO: `/v3/projects/4c151730bf0b4e3aa87965b9fe30c589` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:09.084464 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3e1202-b55a-46a2-95df-a7fdbb328e25 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4c151730bf0b4e3aa87965b9fe30c589)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:09.086410 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3e1202-b55a-46a2-95df-a7fdbb328e25 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:09.130986 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-1e3e1202-b55a-46a2-95df-a7fdbb328e25 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4c151730bf0b4e3aa87965b9fe30c589', 'request_id': 'req-1e3e1202-b55a-46a2-95df-a7fdbb328e25'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:09.139013 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-1e3e1202-b55a-46a2-95df-a7fdbb328e25 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4c151730bf0b4e3aa87965b9fe30c589', 'request_id': 'req-1e3e1202-b55a-46a2-95df-a7fdbb328e25'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:09.144978 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-1e3e1202-b55a-46a2-95df-a7fdbb328e25 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4c151730bf0b4e3aa87965b9fe30c589', 'request_id': 'req-1e3e1202-b55a-46a2-95df-a7fdbb328e25'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:09.155164 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-1e3e1202-b55a-46a2-95df-a7fdbb328e25 admin admin] The token cache is being invalidate because project 4c151730bf0b4e3aa87965b9fe30c589 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81758) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:51:09.155450 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-1e3e1202-b55a-46a2-95df-a7fdbb328e25 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:09.156093 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-1e3e1202-b55a-46a2-95df-a7fdbb328e25 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:09.157618 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1259/2502] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 17:51:09 2022] DELETE /identity/v3/projects/4c151730bf0b4e3aa87965b9fe30c589 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:51:09.162901 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7640b99e-10be-4b53-a998-ea2d1a1f9461 None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:09.163388 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7640b99e-10be-4b53-a998-ea2d1a1f9461 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:09.163498 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7640b99e-10be-4b53-a998-ea2d1a1f9461 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:09.183686 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-7640b99e-10be-4b53-a998-ea2d1a1f9461 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:51:09.184908 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-7640b99e-10be-4b53-a998-ea2d1a1f9461 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:09.246689 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1244/2503] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:51:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:51:09.254491 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cad45d-0781-4316-9ed8-7d8de3224fd3 None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:09.254778 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cad45d-0781-4316-9ed8-7d8de3224fd3 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:09.255114 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cad45d-0781-4316-9ed8-7d8de3224fd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:09.276449 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-60cad45d-0781-4316-9ed8-7d8de3224fd3 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:51:09.277933 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-60cad45d-0781-4316-9ed8-7d8de3224fd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:09.286929 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1260/2504] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:51:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:51:09.291849 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7640b99e-10be-4b53-a998-ea2d1a1f9461 None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:09.304071 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7640b99e-10be-4b53-a998-ea2d1a1f9461 None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:09.309517 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefc0f4b-684f-43ea-b5d2-3f35a23d6749 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:09.310196 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefc0f4b-684f-43ea-b5d2-3f35a23d6749 admin admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:09.310470 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefc0f4b-684f-43ea-b5d2-3f35a23d6749 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:09.310743 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefc0f4b-684f-43ea-b5d2-3f35a23d6749 admin admin] PATH_INFO: `/v3/domains` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:09.311414 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefc0f4b-684f-43ea-b5d2-3f35a23d6749 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:51:09.311717 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefc0f4b-684f-43ea-b5d2-3f35a23d6749 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:09.313379 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefc0f4b-684f-43ea-b5d2-3f35a23d6749 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:09.325157 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1245/2505] 10.222.0.45 () {58 vars in 1153 bytes} [Wed Aug 24 17:51:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:51:09.329481 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cad45d-0781-4316-9ed8-7d8de3224fd3 None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:09.338717 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cad45d-0781-4316-9ed8-7d8de3224fd3 None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:09.343339 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94b531f-adfc-4899-906a-fb91988328db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:09.343932 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94b531f-adfc-4899-906a-fb91988328db admin admin] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:09.344209 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94b531f-adfc-4899-906a-fb91988328db admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:09.344481 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94b531f-adfc-4899-906a-fb91988328db admin admin] PATH_INFO: `/v3/projects` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:09.345133 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94b531f-adfc-4899-906a-fb91988328db admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:09.347019 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94b531f-adfc-4899-906a-fb91988328db admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:09.375081 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1261/2506] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 17:51:09 2022] POST /identity/v3/projects => generated 377 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:51:09.379313 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefc0f4b-684f-43ea-b5d2-3f35a23d6749 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:09.390403 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefc0f4b-684f-43ea-b5d2-3f35a23d6749 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:09.450117 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee89e13f-0c8b-47ce-bc8b-d74abe32b1b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:09.450810 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee89e13f-0c8b-47ce-bc8b-d74abe32b1b9 admin admin] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:09.451093 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee89e13f-0c8b-47ce-bc8b-d74abe32b1b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:09.451362 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee89e13f-0c8b-47ce-bc8b-d74abe32b1b9 admin admin] PATH_INFO: `/v3/users` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:09.452129 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee89e13f-0c8b-47ce-bc8b-d74abe32b1b9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:09.454234 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee89e13f-0c8b-47ce-bc8b-d74abe32b1b9 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:09.477404 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1246/2507] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 17:51:09 2022] POST /identity/v3/users => generated 289 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:51:09.481904 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94b531f-adfc-4899-906a-fb91988328db admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:09.492824 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94b531f-adfc-4899-906a-fb91988328db admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:09.497486 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870a7fcf-f235-4291-a2dc-12d30d8e2ec2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:09.498085 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870a7fcf-f235-4291-a2dc-12d30d8e2ec2 admin admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:09.498369 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870a7fcf-f235-4291-a2dc-12d30d8e2ec2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:09.498646 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870a7fcf-f235-4291-a2dc-12d30d8e2ec2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:09.499292 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870a7fcf-f235-4291-a2dc-12d30d8e2ec2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:09.500938 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870a7fcf-f235-4291-a2dc-12d30d8e2ec2 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:09.511363 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1262/2508] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:51:09 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:51:09.515318 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee89e13f-0c8b-47ce-bc8b-d74abe32b1b9 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:09.525743 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee89e13f-0c8b-47ce-bc8b-d74abe32b1b9 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:09.530341 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649daaa7-3afc-40ca-a247-57b48d330ade admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:09.530929 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649daaa7-3afc-40ca-a247-57b48d330ade admin admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:09.531211 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649daaa7-3afc-40ca-a247-57b48d330ade admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:09.531486 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649daaa7-3afc-40ca-a247-57b48d330ade admin admin] PATH_INFO: `/v3/roles` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:09.532121 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649daaa7-3afc-40ca-a247-57b48d330ade admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:09.533831 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649daaa7-3afc-40ca-a247-57b48d330ade admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:09.535827 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:51:09.545856 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1247/2509] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:51:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 17:51:09.550113 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870a7fcf-f235-4291-a2dc-12d30d8e2ec2 admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:09.559937 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870a7fcf-f235-4291-a2dc-12d30d8e2ec2 admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:09.564833 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b04af9-f53a-4a51-9956-6061f1da10f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:09.565476 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b04af9-f53a-4a51-9956-6061f1da10f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:09.565760 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b04af9-f53a-4a51-9956-6061f1da10f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:09.566039 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b04af9-f53a-4a51-9956-6061f1da10f2 admin admin] PATH_INFO: `/v3/projects/8501ffa9161b426b99853411740cd4a7/users/de2af4615a7e43279bc8672fb1b4e81f/roles/02476a5b82eb4b188c18a5d706a3ef68` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:09.581958 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b04af9-f53a-4a51-9956-6061f1da10f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8501ffa9161b426b99853411740cd4a7, user_id=de2af4615a7e43279bc8672fb1b4e81f, role_id=02476a5b82eb4b188c18a5d706a3ef68)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:09.585315 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b04af9-f53a-4a51-9956-6061f1da10f2 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:09.598171 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1263/2510] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 17:51:09 2022] PUT /identity/v3/projects/8501ffa9161b426b99853411740cd4a7/users/de2af4615a7e43279bc8672fb1b4e81f/roles/02476a5b82eb4b188c18a5d706a3ef68 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:51:09.609615 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919f9718-6841-44a7-b4de-6bc5e8ea44e3 None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:09.609920 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919f9718-6841-44a7-b4de-6bc5e8ea44e3 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:09.610193 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919f9718-6841-44a7-b4de-6bc5e8ea44e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:09.637460 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-919f9718-6841-44a7-b4de-6bc5e8ea44e3 None None] MFA Rules not processed for user `de2af4615a7e43279bc8672fb1b4e81f`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:51:09.638655 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-919f9718-6841-44a7-b4de-6bc5e8ea44e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:09.702566 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1248/2511] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:51:09 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:51:09.706863 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b04af9-f53a-4a51-9956-6061f1da10f2 admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:09.754308 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:51:09.769248 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b04af9-f53a-4a51-9956-6061f1da10f2 admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:09.774655 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157a6f6f-9031-48d8-8e3f-fcbb31c7e9c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:09.775270 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157a6f6f-9031-48d8-8e3f-fcbb31c7e9c1 admin admin] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:09.775551 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157a6f6f-9031-48d8-8e3f-fcbb31c7e9c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:09.775826 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157a6f6f-9031-48d8-8e3f-fcbb31c7e9c1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:09.776458 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157a6f6f-9031-48d8-8e3f-fcbb31c7e9c1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:09.778429 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157a6f6f-9031-48d8-8e3f-fcbb31c7e9c1 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:09.805431 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1264/2512] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 17:51:09 2022] POST /identity/v3/projects => generated 381 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:51:09.809436 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919f9718-6841-44a7-b4de-6bc5e8ea44e3 None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:09.820716 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919f9718-6841-44a7-b4de-6bc5e8ea44e3 None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:09.874443 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a8144f-f86d-476a-831a-7def53126ac5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:09.875105 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a8144f-f86d-476a-831a-7def53126ac5 admin admin] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:09.875386 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a8144f-f86d-476a-831a-7def53126ac5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:09.875658 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a8144f-f86d-476a-831a-7def53126ac5 admin admin] PATH_INFO: `/v3/users` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:09.876287 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a8144f-f86d-476a-831a-7def53126ac5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:09.878255 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a8144f-f86d-476a-831a-7def53126ac5 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:09.899846 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1249/2513] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 17:51:09 2022] POST /identity/v3/users => generated 297 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:51:09.903836 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157a6f6f-9031-48d8-8e3f-fcbb31c7e9c1 admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:09.914385 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157a6f6f-9031-48d8-8e3f-fcbb31c7e9c1 admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:09.919017 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c229293d-180f-49c7-a28b-c73141771304 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:09.919623 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c229293d-180f-49c7-a28b-c73141771304 admin admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:09.919907 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c229293d-180f-49c7-a28b-c73141771304 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:09.920186 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c229293d-180f-49c7-a28b-c73141771304 admin admin] PATH_INFO: `/v3/roles` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:09.920870 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c229293d-180f-49c7-a28b-c73141771304 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:09.922456 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c229293d-180f-49c7-a28b-c73141771304 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:09.933428 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1265/2514] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:51:09 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:51:09.937516 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a8144f-f86d-476a-831a-7def53126ac5 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:09.948060 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a8144f-f86d-476a-831a-7def53126ac5 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:09.952773 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb9ad7a-903f-480f-a517-97e5018bc1f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:09.953369 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb9ad7a-903f-480f-a517-97e5018bc1f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:09.953654 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb9ad7a-903f-480f-a517-97e5018bc1f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:09.953934 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb9ad7a-903f-480f-a517-97e5018bc1f9 admin admin] PATH_INFO: `/v3/projects/0ba8db58eeab454ebb90b65fcccf6e14/users/edb7b98ad61f4e89959b978b3c38d184/roles/e411ad9eb96e4e8d85b59b5b700ada7b` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:09.969457 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb9ad7a-903f-480f-a517-97e5018bc1f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ba8db58eeab454ebb90b65fcccf6e14, user_id=edb7b98ad61f4e89959b978b3c38d184, role_id=e411ad9eb96e4e8d85b59b5b700ada7b)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:09.972152 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb9ad7a-903f-480f-a517-97e5018bc1f9 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:09.984556 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1250/2515] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 17:51:09 2022] PUT /identity/v3/projects/0ba8db58eeab454ebb90b65fcccf6e14/users/edb7b98ad61f4e89959b978b3c38d184/roles/e411ad9eb96e4e8d85b59b5b700ada7b => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:51:09.988439 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c229293d-180f-49c7-a28b-c73141771304 admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:09.999116 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c229293d-180f-49c7-a28b-c73141771304 admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:10.055650 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e98d6c-6297-48ad-8f39-0a256b075c45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:10.056283 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e98d6c-6297-48ad-8f39-0a256b075c45 admin admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:10.056528 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e98d6c-6297-48ad-8f39-0a256b075c45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:10.056875 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e98d6c-6297-48ad-8f39-0a256b075c45 admin admin] PATH_INFO: `/v3/roles` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:10.057520 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e98d6c-6297-48ad-8f39-0a256b075c45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:10.059114 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e98d6c-6297-48ad-8f39-0a256b075c45 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:10.068906 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1266/2516] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:51:09 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 17:51:10.073127 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb9ad7a-903f-480f-a517-97e5018bc1f9 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:10.083147 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb9ad7a-903f-480f-a517-97e5018bc1f9 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:10.087731 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb77418-83de-4361-8cb2-862d89d12f82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:10.088367 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb77418-83de-4361-8cb2-862d89d12f82 admin admin] REQUEST_METHOD: `PUT` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:10.088683 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb77418-83de-4361-8cb2-862d89d12f82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:10.088955 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb77418-83de-4361-8cb2-862d89d12f82 admin admin] PATH_INFO: `/v3/projects/0ba8db58eeab454ebb90b65fcccf6e14/users/edb7b98ad61f4e89959b978b3c38d184/roles/e411ad9eb96e4e8d85b59b5b700ada7b` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:10.090865 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb77418-83de-4361-8cb2-862d89d12f82 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ba8db58eeab454ebb90b65fcccf6e14, user_id=edb7b98ad61f4e89959b978b3c38d184, role_id=e411ad9eb96e4e8d85b59b5b700ada7b)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:10.093510 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb77418-83de-4361-8cb2-862d89d12f82 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:10.103916 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1251/2517] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 17:51:10 2022] PUT /identity/v3/projects/0ba8db58eeab454ebb90b65fcccf6e14/users/edb7b98ad61f4e89959b978b3c38d184/roles/e411ad9eb96e4e8d85b59b5b700ada7b => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:51:10.113774 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24ba982-16ba-4de3-884b-ecc7c5617ad9 None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:10.114061 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24ba982-16ba-4de3-884b-ecc7c5617ad9 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:10.114388 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24ba982-16ba-4de3-884b-ecc7c5617ad9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:10.141424 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-d24ba982-16ba-4de3-884b-ecc7c5617ad9 None None] MFA Rules not processed for user `edb7b98ad61f4e89959b978b3c38d184`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:51:10.142628 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-d24ba982-16ba-4de3-884b-ecc7c5617ad9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:10.208165 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1267/2518] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:51:10 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:51:10.220242 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb77418-83de-4361-8cb2-862d89d12f82 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:10.221969 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-7bb77418-83de-4361-8cb2-862d89d12f82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:10.265850 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:51:10.280762 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb77418-83de-4361-8cb2-862d89d12f82 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:10.284310 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd67ed7-79a3-458c-9502-70ea8d8cb057 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59be342541264942ad87f60ff4de6c8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:10.285004 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd67ed7-79a3-458c-9502-70ea8d8cb057 service cinder] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:10.285280 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd67ed7-79a3-458c-9502-70ea8d8cb057 service cinder] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:10.285549 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd67ed7-79a3-458c-9502-70ea8d8cb057 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:10.295528 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd67ed7-79a3-458c-9502-70ea8d8cb057 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:10.297036 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd67ed7-79a3-458c-9502-70ea8d8cb057 service cinder] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:10.306914 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1252/2519] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:51:10 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:51:10.611845 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24ba982-16ba-4de3-884b-ecc7c5617ad9 None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:10.621221 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24ba982-16ba-4de3-884b-ecc7c5617ad9 None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:10.625992 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d997b6e-feab-4d81-b4d2-e8d1100e60c6 tempest-VolumeQuotaClassesTest-1258981584 tempest-VolumeQuotaClassesTest-1258981584-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edb7b98ad61f4e89959b978b3c38d184', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ba8db58eeab454ebb90b65fcccf6e14', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:10.626602 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d997b6e-feab-4d81-b4d2-e8d1100e60c6 tempest-VolumeQuotaClassesTest-1258981584 tempest-VolumeQuotaClassesTest-1258981584-project-admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:10.626904 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d997b6e-feab-4d81-b4d2-e8d1100e60c6 tempest-VolumeQuotaClassesTest-1258981584 tempest-VolumeQuotaClassesTest-1258981584-project-admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:10.627164 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d997b6e-feab-4d81-b4d2-e8d1100e60c6 tempest-VolumeQuotaClassesTest-1258981584 tempest-VolumeQuotaClassesTest-1258981584-project-admin] PATH_INFO: `/v3/domains` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:10.627815 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d997b6e-feab-4d81-b4d2-e8d1100e60c6 tempest-VolumeQuotaClassesTest-1258981584 tempest-VolumeQuotaClassesTest-1258981584-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:51:10.628122 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d997b6e-feab-4d81-b4d2-e8d1100e60c6 tempest-VolumeQuotaClassesTest-1258981584 tempest-VolumeQuotaClassesTest-1258981584-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:10.629762 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d997b6e-feab-4d81-b4d2-e8d1100e60c6 tempest-VolumeQuotaClassesTest-1258981584 tempest-VolumeQuotaClassesTest-1258981584-project-admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:10.641184 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1268/2520] 10.222.0.45 () {58 vars in 1153 bytes} [Wed Aug 24 17:51:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:51:10.645299 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd67ed7-79a3-458c-9502-70ea8d8cb057 service cinder] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:10.654729 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd67ed7-79a3-458c-9502-70ea8d8cb057 service cinder] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:10.659203 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3909c9f-4fb6-4b7d-8b0b-62292114cb3a tempest-VolumeQuotaClassesTest-1258981584 tempest-VolumeQuotaClassesTest-1258981584-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edb7b98ad61f4e89959b978b3c38d184', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ba8db58eeab454ebb90b65fcccf6e14', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:10.659794 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3909c9f-4fb6-4b7d-8b0b-62292114cb3a tempest-VolumeQuotaClassesTest-1258981584 tempest-VolumeQuotaClassesTest-1258981584-project-admin] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:10.660078 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3909c9f-4fb6-4b7d-8b0b-62292114cb3a tempest-VolumeQuotaClassesTest-1258981584 tempest-VolumeQuotaClassesTest-1258981584-project-admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:10.660348 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3909c9f-4fb6-4b7d-8b0b-62292114cb3a tempest-VolumeQuotaClassesTest-1258981584 tempest-VolumeQuotaClassesTest-1258981584-project-admin] PATH_INFO: `/v3/projects` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:10.661042 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3909c9f-4fb6-4b7d-8b0b-62292114cb3a tempest-VolumeQuotaClassesTest-1258981584 tempest-VolumeQuotaClassesTest-1258981584-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:10.663847 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3909c9f-4fb6-4b7d-8b0b-62292114cb3a tempest-VolumeQuotaClassesTest-1258981584 tempest-VolumeQuotaClassesTest-1258981584-project-admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:10.689542 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1253/2521] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 17:51:10 2022] POST /identity/v3/projects => generated 354 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:51:10.693610 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d997b6e-feab-4d81-b4d2-e8d1100e60c6 tempest-VolumeQuotaClassesTest-1258981584 tempest-VolumeQuotaClassesTest-1258981584-project-admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:10.703294 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d997b6e-feab-4d81-b4d2-e8d1100e60c6 tempest-VolumeQuotaClassesTest-1258981584 tempest-VolumeQuotaClassesTest-1258981584-project-admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:10.732277 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:51:10.759708 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c739a14d-6ed9-48d1-b316-3fa3f626903e tempest-VolumeQuotaClassesTest-1258981584 tempest-VolumeQuotaClassesTest-1258981584-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edb7b98ad61f4e89959b978b3c38d184', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ba8db58eeab454ebb90b65fcccf6e14', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:10.760309 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c739a14d-6ed9-48d1-b316-3fa3f626903e tempest-VolumeQuotaClassesTest-1258981584 tempest-VolumeQuotaClassesTest-1258981584-project-admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:10.760682 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c739a14d-6ed9-48d1-b316-3fa3f626903e tempest-VolumeQuotaClassesTest-1258981584 tempest-VolumeQuotaClassesTest-1258981584-project-admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:10.760965 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c739a14d-6ed9-48d1-b316-3fa3f626903e tempest-VolumeQuotaClassesTest-1258981584 tempest-VolumeQuotaClassesTest-1258981584-project-admin] PATH_INFO: `/v3/domains` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:10.761605 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c739a14d-6ed9-48d1-b316-3fa3f626903e tempest-VolumeQuotaClassesTest-1258981584 tempest-VolumeQuotaClassesTest-1258981584-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:51:10.761911 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c739a14d-6ed9-48d1-b316-3fa3f626903e tempest-VolumeQuotaClassesTest-1258981584 tempest-VolumeQuotaClassesTest-1258981584-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:10.763494 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c739a14d-6ed9-48d1-b316-3fa3f626903e tempest-VolumeQuotaClassesTest-1258981584 tempest-VolumeQuotaClassesTest-1258981584-project-admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:10.776093 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1269/2522] 10.222.0.45 () {58 vars in 1153 bytes} [Wed Aug 24 17:51:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:51:10.812524 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3909c9f-4fb6-4b7d-8b0b-62292114cb3a tempest-VolumeQuotaClassesTest-1258981584 tempest-VolumeQuotaClassesTest-1258981584-project-admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:10.823334 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3909c9f-4fb6-4b7d-8b0b-62292114cb3a tempest-VolumeQuotaClassesTest-1258981584 tempest-VolumeQuotaClassesTest-1258981584-project-admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:10.827862 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede28244-5046-406e-92a4-6fcc0764bfab tempest-VolumeQuotaClassesTest-1258981584 tempest-VolumeQuotaClassesTest-1258981584-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edb7b98ad61f4e89959b978b3c38d184', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ba8db58eeab454ebb90b65fcccf6e14', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:10.828484 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede28244-5046-406e-92a4-6fcc0764bfab tempest-VolumeQuotaClassesTest-1258981584 tempest-VolumeQuotaClassesTest-1258981584-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:10.828800 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede28244-5046-406e-92a4-6fcc0764bfab tempest-VolumeQuotaClassesTest-1258981584 tempest-VolumeQuotaClassesTest-1258981584-project-admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:10.829077 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede28244-5046-406e-92a4-6fcc0764bfab tempest-VolumeQuotaClassesTest-1258981584 tempest-VolumeQuotaClassesTest-1258981584-project-admin] PATH_INFO: `/v3/projects/5127c99369724535afe942b11e439162` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:10.830207 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede28244-5046-406e-92a4-6fcc0764bfab tempest-VolumeQuotaClassesTest-1258981584 tempest-VolumeQuotaClassesTest-1258981584-project-admin] RBAC: Authorizing `identity:delete_project(project_id=5127c99369724535afe942b11e439162)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:10.832104 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede28244-5046-406e-92a4-6fcc0764bfab tempest-VolumeQuotaClassesTest-1258981584 tempest-VolumeQuotaClassesTest-1258981584-project-admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:10.876977 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-ede28244-5046-406e-92a4-6fcc0764bfab tempest-VolumeQuotaClassesTest-1258981584 tempest-VolumeQuotaClassesTest-1258981584-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5127c99369724535afe942b11e439162', 'request_id': 'req-ede28244-5046-406e-92a4-6fcc0764bfab'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:10.883017 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-ede28244-5046-406e-92a4-6fcc0764bfab tempest-VolumeQuotaClassesTest-1258981584 tempest-VolumeQuotaClassesTest-1258981584-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5127c99369724535afe942b11e439162', 'request_id': 'req-ede28244-5046-406e-92a4-6fcc0764bfab'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:10.889657 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-ede28244-5046-406e-92a4-6fcc0764bfab tempest-VolumeQuotaClassesTest-1258981584 tempest-VolumeQuotaClassesTest-1258981584-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5127c99369724535afe942b11e439162', 'request_id': 'req-ede28244-5046-406e-92a4-6fcc0764bfab'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:10.903108 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-ede28244-5046-406e-92a4-6fcc0764bfab tempest-VolumeQuotaClassesTest-1258981584 tempest-VolumeQuotaClassesTest-1258981584-project-admin] The token cache is being invalidate because project 5127c99369724535afe942b11e439162 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81759) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:51:10.903394 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-ede28244-5046-406e-92a4-6fcc0764bfab tempest-VolumeQuotaClassesTest-1258981584 tempest-VolumeQuotaClassesTest-1258981584-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:10.904021 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-ede28244-5046-406e-92a4-6fcc0764bfab tempest-VolumeQuotaClassesTest-1258981584 tempest-VolumeQuotaClassesTest-1258981584-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:10.905547 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1254/2523] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 17:51:10 2022] DELETE /identity/v3/projects/5127c99369724535afe942b11e439162 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:51:11.092441 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c739a14d-6ed9-48d1-b316-3fa3f626903e tempest-VolumeQuotaClassesTest-1258981584 tempest-VolumeQuotaClassesTest-1258981584-project-admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:11.094229 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-c739a14d-6ed9-48d1-b316-3fa3f626903e tempest-VolumeQuotaClassesTest-1258981584 tempest-VolumeQuotaClassesTest-1258981584-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:11.158552 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c739a14d-6ed9-48d1-b316-3fa3f626903e tempest-VolumeQuotaClassesTest-1258981584 tempest-VolumeQuotaClassesTest-1258981584-project-admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:11.214639 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f57f05-4656-4701-8111-8857540efd8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:11.215271 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f57f05-4656-4701-8111-8857540efd8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:11.215552 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f57f05-4656-4701-8111-8857540efd8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:11.215830 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f57f05-4656-4701-8111-8857540efd8f admin admin] PATH_INFO: `/v3/users/de2af4615a7e43279bc8672fb1b4e81f` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:11.216974 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f57f05-4656-4701-8111-8857540efd8f admin admin] RBAC: Authorizing `identity:delete_user(user_id=de2af4615a7e43279bc8672fb1b4e81f)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:11.218901 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f57f05-4656-4701-8111-8857540efd8f admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:11.263136 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-a3f57f05-4656-4701-8111-8857540efd8f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de2af4615a7e43279bc8672fb1b4e81f'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:11.263777 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-a3f57f05-4656-4701-8111-8857540efd8f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de2af4615a7e43279bc8672fb1b4e81f'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:11.270425 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-a3f57f05-4656-4701-8111-8857540efd8f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de2af4615a7e43279bc8672fb1b4e81f'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:11.271064 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-a3f57f05-4656-4701-8111-8857540efd8f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de2af4615a7e43279bc8672fb1b4e81f'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:11.282472 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-a3f57f05-4656-4701-8111-8857540efd8f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de2af4615a7e43279bc8672fb1b4e81f'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:11.284092 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:51:11.298132 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1270/2524] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 17:51:11 2022] DELETE /identity/v3/users/de2af4615a7e43279bc8672fb1b4e81f => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:51:11.303220 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4d39c0-5a27-4f51-9df4-e1f1218b9173 None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:11.303678 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4d39c0-5a27-4f51-9df4-e1f1218b9173 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:11.303810 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4d39c0-5a27-4f51-9df4-e1f1218b9173 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:11.323653 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-1b4d39c0-5a27-4f51-9df4-e1f1218b9173 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:51:11.324889 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-1b4d39c0-5a27-4f51-9df4-e1f1218b9173 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:11.381082 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1255/2525] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:51:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:51:11.390548 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f57f05-4656-4701-8111-8857540efd8f admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:11.391998 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-a3f57f05-4656-4701-8111-8857540efd8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:11.450634 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f57f05-4656-4701-8111-8857540efd8f admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:11.454363 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3f0270-618f-41bb-a5a6-b51ca24909f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43ad98a250a144099ccd18378aedd4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:11.455041 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3f0270-618f-41bb-a5a6-b51ca24909f2 service neutron] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:11.455392 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3f0270-618f-41bb-a5a6-b51ca24909f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:11.455993 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3f0270-618f-41bb-a5a6-b51ca24909f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:11.466275 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3f0270-618f-41bb-a5a6-b51ca24909f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:11.467733 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3f0270-618f-41bb-a5a6-b51ca24909f2 service neutron] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:11.478227 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1271/2526] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:51:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:51:12.054136 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4d39c0-5a27-4f51-9df4-e1f1218b9173 None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:12.055607 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-1b4d39c0-5a27-4f51-9df4-e1f1218b9173 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:12.067774 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4d39c0-5a27-4f51-9df4-e1f1218b9173 None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:12.083712 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c861a03-88f9-4354-ae56-9c03d93d6de8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:12.084485 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c861a03-88f9-4354-ae56-9c03d93d6de8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:12.084757 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c861a03-88f9-4354-ae56-9c03d93d6de8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:12.085114 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c861a03-88f9-4354-ae56-9c03d93d6de8 admin admin] PATH_INFO: `/v3/projects/8501ffa9161b426b99853411740cd4a7` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:12.086272 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c861a03-88f9-4354-ae56-9c03d93d6de8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8501ffa9161b426b99853411740cd4a7)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:12.088259 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c861a03-88f9-4354-ae56-9c03d93d6de8 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:12.134549 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:51:12.142984 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-1c861a03-88f9-4354-ae56-9c03d93d6de8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8501ffa9161b426b99853411740cd4a7', 'request_id': 'req-1c861a03-88f9-4354-ae56-9c03d93d6de8'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:12.149669 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-1c861a03-88f9-4354-ae56-9c03d93d6de8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8501ffa9161b426b99853411740cd4a7', 'request_id': 'req-1c861a03-88f9-4354-ae56-9c03d93d6de8'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:12.156294 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-1c861a03-88f9-4354-ae56-9c03d93d6de8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8501ffa9161b426b99853411740cd4a7', 'request_id': 'req-1c861a03-88f9-4354-ae56-9c03d93d6de8'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:12.169882 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-1c861a03-88f9-4354-ae56-9c03d93d6de8 admin admin] The token cache is being invalidate because project 8501ffa9161b426b99853411740cd4a7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81759) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:51:12.170176 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-1c861a03-88f9-4354-ae56-9c03d93d6de8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:12.170809 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-1c861a03-88f9-4354-ae56-9c03d93d6de8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:12.172317 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1256/2527] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 17:51:12 2022] DELETE /identity/v3/projects/8501ffa9161b426b99853411740cd4a7 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:51:12.176517 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3f0270-618f-41bb-a5a6-b51ca24909f2 service neutron] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:12.186995 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3f0270-618f-41bb-a5a6-b51ca24909f2 service neutron] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:12.243334 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2addaa-56e8-49bc-acbe-9b529399764c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:12.243970 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2addaa-56e8-49bc-acbe-9b529399764c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:12.244248 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2addaa-56e8-49bc-acbe-9b529399764c admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:12.244523 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2addaa-56e8-49bc-acbe-9b529399764c admin admin] PATH_INFO: `/v3/users/edb7b98ad61f4e89959b978b3c38d184` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:12.245704 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2addaa-56e8-49bc-acbe-9b529399764c admin admin] RBAC: Authorizing `identity:delete_user(user_id=edb7b98ad61f4e89959b978b3c38d184)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:12.247677 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2addaa-56e8-49bc-acbe-9b529399764c admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:12.300856 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-3f2addaa-56e8-49bc-acbe-9b529399764c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'edb7b98ad61f4e89959b978b3c38d184'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:12.301557 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-3f2addaa-56e8-49bc-acbe-9b529399764c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'edb7b98ad61f4e89959b978b3c38d184'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:12.308662 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-3f2addaa-56e8-49bc-acbe-9b529399764c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'edb7b98ad61f4e89959b978b3c38d184'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:12.309304 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-3f2addaa-56e8-49bc-acbe-9b529399764c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'edb7b98ad61f4e89959b978b3c38d184'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:12.321440 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-3f2addaa-56e8-49bc-acbe-9b529399764c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'edb7b98ad61f4e89959b978b3c38d184'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:12.335466 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1272/2528] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 17:51:12 2022] DELETE /identity/v3/users/edb7b98ad61f4e89959b978b3c38d184 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:51:12.619631 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c861a03-88f9-4354-ae56-9c03d93d6de8 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:12.624598 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-1c861a03-88f9-4354-ae56-9c03d93d6de8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:12.640757 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c861a03-88f9-4354-ae56-9c03d93d6de8 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:12.727035 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33ec543-a99c-4825-97f4-410422e45279 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:12.727814 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33ec543-a99c-4825-97f4-410422e45279 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:12.728101 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33ec543-a99c-4825-97f4-410422e45279 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:12.728382 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33ec543-a99c-4825-97f4-410422e45279 admin admin] PATH_INFO: `/v3/projects/0ba8db58eeab454ebb90b65fcccf6e14` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:12.729584 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33ec543-a99c-4825-97f4-410422e45279 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ba8db58eeab454ebb90b65fcccf6e14)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:12.731497 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33ec543-a99c-4825-97f4-410422e45279 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:12.804391 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-f33ec543-a99c-4825-97f4-410422e45279 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ba8db58eeab454ebb90b65fcccf6e14', 'request_id': 'req-f33ec543-a99c-4825-97f4-410422e45279'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:12.811055 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-f33ec543-a99c-4825-97f4-410422e45279 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ba8db58eeab454ebb90b65fcccf6e14', 'request_id': 'req-f33ec543-a99c-4825-97f4-410422e45279'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:12.819549 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-f33ec543-a99c-4825-97f4-410422e45279 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ba8db58eeab454ebb90b65fcccf6e14', 'request_id': 'req-f33ec543-a99c-4825-97f4-410422e45279'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:12.835842 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-f33ec543-a99c-4825-97f4-410422e45279 admin admin] The token cache is being invalidate because project 0ba8db58eeab454ebb90b65fcccf6e14 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81759) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:51:12.836213 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-f33ec543-a99c-4825-97f4-410422e45279 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:12.836949 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-f33ec543-a99c-4825-97f4-410422e45279 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:12.838839 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1257/2529] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 17:51:12 2022] DELETE /identity/v3/projects/0ba8db58eeab454ebb90b65fcccf6e14 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:51:12.843547 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b00829-a688-4c25-968c-09a4495e7f83 None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:12.844020 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b00829-a688-4c25-968c-09a4495e7f83 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:12.844301 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b00829-a688-4c25-968c-09a4495e7f83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:12.865520 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-24b00829-a688-4c25-968c-09a4495e7f83 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:51:12.867453 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-24b00829-a688-4c25-968c-09a4495e7f83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:12.878859 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:INSERT=2,keystone:DELETE=11,keystone:SELECT=87 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:51:12.945114 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1273/2530] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:51:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:51:12.950257 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaf6da5-f0fa-468b-bfc3-8cb860440ce8 None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:12.950545 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaf6da5-f0fa-468b-bfc3-8cb860440ce8 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:12.950825 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaf6da5-f0fa-468b-bfc3-8cb860440ce8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:12.971432 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-abaf6da5-f0fa-468b-bfc3-8cb860440ce8 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:51:12.972748 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-abaf6da5-f0fa-468b-bfc3-8cb860440ce8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:12.981333 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1258/2531] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:51:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:51:12.985901 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b00829-a688-4c25-968c-09a4495e7f83 None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:12.997491 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b00829-a688-4c25-968c-09a4495e7f83 None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:13.003444 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a511ca72-6837-4035-a44d-c2965974736e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:13.004300 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a511ca72-6837-4035-a44d-c2965974736e admin admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:13.004806 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a511ca72-6837-4035-a44d-c2965974736e admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:13.006101 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a511ca72-6837-4035-a44d-c2965974736e admin admin] PATH_INFO: `/v3/domains` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:13.006847 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a511ca72-6837-4035-a44d-c2965974736e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:51:13.007225 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a511ca72-6837-4035-a44d-c2965974736e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:13.008957 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a511ca72-6837-4035-a44d-c2965974736e admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:13.028885 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1274/2532] 10.222.0.45 () {58 vars in 1153 bytes} [Wed Aug 24 17:51:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:51:13.030761 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abaf6da5-f0fa-468b-bfc3-8cb860440ce8 None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:13.045812 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abaf6da5-f0fa-468b-bfc3-8cb860440ce8 None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:13.051161 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da010c86-4839-4aec-a7c6-5dc7d694dcbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:13.052438 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da010c86-4839-4aec-a7c6-5dc7d694dcbe admin admin] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:13.052865 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da010c86-4839-4aec-a7c6-5dc7d694dcbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:13.053239 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da010c86-4839-4aec-a7c6-5dc7d694dcbe admin admin] PATH_INFO: `/v3/projects` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:13.053978 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da010c86-4839-4aec-a7c6-5dc7d694dcbe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:13.055989 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da010c86-4839-4aec-a7c6-5dc7d694dcbe admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:13.085300 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1259/2533] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 17:51:13 2022] POST /identity/v3/projects => generated 391 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:51:13.089425 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a511ca72-6837-4035-a44d-c2965974736e admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:13.100369 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a511ca72-6837-4035-a44d-c2965974736e admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:13.157765 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c063dff8-75c9-4a27-9f17-bb3bd2febb17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:13.158573 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c063dff8-75c9-4a27-9f17-bb3bd2febb17 admin admin] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:13.158948 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c063dff8-75c9-4a27-9f17-bb3bd2febb17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:13.159318 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c063dff8-75c9-4a27-9f17-bb3bd2febb17 admin admin] PATH_INFO: `/v3/users` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:13.160065 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c063dff8-75c9-4a27-9f17-bb3bd2febb17 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:13.163042 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c063dff8-75c9-4a27-9f17-bb3bd2febb17 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:13.191480 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1275/2534] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 17:51:13 2022] POST /identity/v3/users => generated 296 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:51:13.192997 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da010c86-4839-4aec-a7c6-5dc7d694dcbe admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:13.204320 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da010c86-4839-4aec-a7c6-5dc7d694dcbe admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:13.209062 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6eec872-faba-45b6-80a6-383a97aef987 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:13.209669 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6eec872-faba-45b6-80a6-383a97aef987 admin admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:13.209946 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6eec872-faba-45b6-80a6-383a97aef987 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:13.210214 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6eec872-faba-45b6-80a6-383a97aef987 admin admin] PATH_INFO: `/v3/roles` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:13.210858 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6eec872-faba-45b6-80a6-383a97aef987 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:13.212451 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6eec872-faba-45b6-80a6-383a97aef987 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:13.222573 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1260/2535] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:51:13 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:51:13.226898 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c063dff8-75c9-4a27-9f17-bb3bd2febb17 admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:13.236591 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c063dff8-75c9-4a27-9f17-bb3bd2febb17 admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:13.241887 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582b83d6-8ce1-4d93-83ec-5215630b1e40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:13.242486 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582b83d6-8ce1-4d93-83ec-5215630b1e40 admin admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:13.242764 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582b83d6-8ce1-4d93-83ec-5215630b1e40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:13.243033 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582b83d6-8ce1-4d93-83ec-5215630b1e40 admin admin] PATH_INFO: `/v3/roles` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:13.243675 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582b83d6-8ce1-4d93-83ec-5215630b1e40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:13.245304 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582b83d6-8ce1-4d93-83ec-5215630b1e40 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:13.256264 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1276/2536] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:51:13 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:51:13.260234 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6eec872-faba-45b6-80a6-383a97aef987 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:13.271494 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6eec872-faba-45b6-80a6-383a97aef987 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:13.276112 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5d9e84-de2b-403e-b4f7-c2c5cf78bf4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:13.276808 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5d9e84-de2b-403e-b4f7-c2c5cf78bf4b admin admin] REQUEST_METHOD: `PUT` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:13.277089 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5d9e84-de2b-403e-b4f7-c2c5cf78bf4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:13.277356 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5d9e84-de2b-403e-b4f7-c2c5cf78bf4b admin admin] PATH_INFO: `/v3/projects/782d84bd35bf4a869c9054dcc87293cb/users/eeb19c1e1d1148f4a0c5ac64393afec4/roles/02476a5b82eb4b188c18a5d706a3ef68` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:13.294743 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5d9e84-de2b-403e-b4f7-c2c5cf78bf4b admin admin] RBAC: Authorizing `identity:create_grant(project_id=782d84bd35bf4a869c9054dcc87293cb, user_id=eeb19c1e1d1148f4a0c5ac64393afec4, role_id=02476a5b82eb4b188c18a5d706a3ef68)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:13.297565 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5d9e84-de2b-403e-b4f7-c2c5cf78bf4b admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:13.300274 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:51:13.313103 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1261/2537] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 17:51:13 2022] PUT /identity/v3/projects/782d84bd35bf4a869c9054dcc87293cb/users/eeb19c1e1d1148f4a0c5ac64393afec4/roles/02476a5b82eb4b188c18a5d706a3ef68 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:51:13.324322 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bbcede-3ec7-4a7b-9d0c-8f022ac28d36 None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:13.324652 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bbcede-3ec7-4a7b-9d0c-8f022ac28d36 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:13.324931 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bbcede-3ec7-4a7b-9d0c-8f022ac28d36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:13.362370 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-33bbcede-3ec7-4a7b-9d0c-8f022ac28d36 None None] MFA Rules not processed for user `eeb19c1e1d1148f4a0c5ac64393afec4`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:51:13.363586 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-33bbcede-3ec7-4a7b-9d0c-8f022ac28d36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:13.434150 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1277/2538] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:51:13 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:51:13.438842 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5d9e84-de2b-403e-b4f7-c2c5cf78bf4b admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:13.512757 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5d9e84-de2b-403e-b4f7-c2c5cf78bf4b admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:13.519605 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172fa3c5-4e42-4c14-a0f0-43ede4f0dce9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:13.519901 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172fa3c5-4e42-4c14-a0f0-43ede4f0dce9 admin admin] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:13.520270 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172fa3c5-4e42-4c14-a0f0-43ede4f0dce9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:13.521282 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172fa3c5-4e42-4c14-a0f0-43ede4f0dce9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:13.522135 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172fa3c5-4e42-4c14-a0f0-43ede4f0dce9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:13.524074 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172fa3c5-4e42-4c14-a0f0-43ede4f0dce9 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:13.554327 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1262/2539] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 17:51:13 2022] POST /identity/v3/projects => generated 393 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:51:13.558349 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bbcede-3ec7-4a7b-9d0c-8f022ac28d36 None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:13.569364 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bbcede-3ec7-4a7b-9d0c-8f022ac28d36 None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:13.574182 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:51:13.634656 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410ac367-9411-4c47-a7b3-19f63c875e8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:13.635401 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410ac367-9411-4c47-a7b3-19f63c875e8b admin admin] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:13.636951 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410ac367-9411-4c47-a7b3-19f63c875e8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:13.637326 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410ac367-9411-4c47-a7b3-19f63c875e8b admin admin] PATH_INFO: `/v3/users` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:13.638111 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410ac367-9411-4c47-a7b3-19f63c875e8b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:13.640459 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410ac367-9411-4c47-a7b3-19f63c875e8b admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:13.673082 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1278/2540] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 17:51:13 2022] POST /identity/v3/users => generated 303 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:51:13.674330 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172fa3c5-4e42-4c14-a0f0-43ede4f0dce9 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:13.687506 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172fa3c5-4e42-4c14-a0f0-43ede4f0dce9 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:13.692624 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1fd009-e700-48b9-8069-93a43dec727e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:13.693314 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1fd009-e700-48b9-8069-93a43dec727e admin admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:13.693668 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1fd009-e700-48b9-8069-93a43dec727e admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:13.694016 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1fd009-e700-48b9-8069-93a43dec727e admin admin] PATH_INFO: `/v3/roles` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:13.695077 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1fd009-e700-48b9-8069-93a43dec727e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:13.697182 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1fd009-e700-48b9-8069-93a43dec727e admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:13.713051 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1263/2541] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:51:13 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 17:51:13.714852 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410ac367-9411-4c47-a7b3-19f63c875e8b admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:13.728512 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410ac367-9411-4c47-a7b3-19f63c875e8b admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:13.733434 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb245f07-ade9-47d5-8b69-17cb7da01dda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:13.734593 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb245f07-ade9-47d5-8b69-17cb7da01dda admin admin] REQUEST_METHOD: `PUT` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:13.735011 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb245f07-ade9-47d5-8b69-17cb7da01dda admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:13.735362 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb245f07-ade9-47d5-8b69-17cb7da01dda admin admin] PATH_INFO: `/v3/projects/c07a57edc321471eb53f70b54047d2f7/users/7ce5dac03e3f48db96d697b33bef94ac/roles/e411ad9eb96e4e8d85b59b5b700ada7b` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:13.753637 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb245f07-ade9-47d5-8b69-17cb7da01dda admin admin] RBAC: Authorizing `identity:create_grant(project_id=c07a57edc321471eb53f70b54047d2f7, user_id=7ce5dac03e3f48db96d697b33bef94ac, role_id=e411ad9eb96e4e8d85b59b5b700ada7b)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:13.757227 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb245f07-ade9-47d5-8b69-17cb7da01dda admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:13.776697 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1279/2542] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 17:51:13 2022] PUT /identity/v3/projects/c07a57edc321471eb53f70b54047d2f7/users/7ce5dac03e3f48db96d697b33bef94ac/roles/e411ad9eb96e4e8d85b59b5b700ada7b => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:51:13.779526 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1fd009-e700-48b9-8069-93a43dec727e admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:13.799072 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1fd009-e700-48b9-8069-93a43dec727e admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:13.866716 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b57e8e-4b1c-4abc-9210-bc8fd222990e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:13.867490 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b57e8e-4b1c-4abc-9210-bc8fd222990e admin admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:13.867967 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b57e8e-4b1c-4abc-9210-bc8fd222990e admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:13.868340 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b57e8e-4b1c-4abc-9210-bc8fd222990e admin admin] PATH_INFO: `/v3/roles` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:13.869266 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b57e8e-4b1c-4abc-9210-bc8fd222990e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:13.870875 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b57e8e-4b1c-4abc-9210-bc8fd222990e admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:13.890521 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1264/2543] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:51:13 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:51:13.891965 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb245f07-ade9-47d5-8b69-17cb7da01dda admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:13.904256 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb245f07-ade9-47d5-8b69-17cb7da01dda admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:13.910151 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d7e0b5-731e-4b7b-b325-84376854d32a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:13.910758 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d7e0b5-731e-4b7b-b325-84376854d32a admin admin] REQUEST_METHOD: `PUT` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:13.911106 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d7e0b5-731e-4b7b-b325-84376854d32a admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:13.911424 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d7e0b5-731e-4b7b-b325-84376854d32a admin admin] PATH_INFO: `/v3/projects/c07a57edc321471eb53f70b54047d2f7/users/7ce5dac03e3f48db96d697b33bef94ac/roles/e411ad9eb96e4e8d85b59b5b700ada7b` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:13.913768 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d7e0b5-731e-4b7b-b325-84376854d32a admin admin] RBAC: Authorizing `identity:create_grant(project_id=c07a57edc321471eb53f70b54047d2f7, user_id=7ce5dac03e3f48db96d697b33bef94ac, role_id=e411ad9eb96e4e8d85b59b5b700ada7b)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:13.916659 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d7e0b5-731e-4b7b-b325-84376854d32a admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:13.937727 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1280/2544] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 17:51:13 2022] PUT /identity/v3/projects/c07a57edc321471eb53f70b54047d2f7/users/7ce5dac03e3f48db96d697b33bef94ac/roles/e411ad9eb96e4e8d85b59b5b700ada7b => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:51:13.940342 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7022dd8c-8f3a-42fa-9d71-bcf09885b27b None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:13.940723 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7022dd8c-8f3a-42fa-9d71-bcf09885b27b None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:13.941052 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7022dd8c-8f3a-42fa-9d71-bcf09885b27b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:13.975756 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-7022dd8c-8f3a-42fa-9d71-bcf09885b27b None None] MFA Rules not processed for user `7ce5dac03e3f48db96d697b33bef94ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:51:13.977182 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-7022dd8c-8f3a-42fa-9d71-bcf09885b27b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:14.037408 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:51:14.060397 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1265/2545] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:51:13 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:51:14.073397 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d7e0b5-731e-4b7b-b325-84376854d32a admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:14.075691 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-10d7e0b5-731e-4b7b-b325-84376854d32a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:14.147726 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d7e0b5-731e-4b7b-b325-84376854d32a admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:14.151787 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8df5a53-2e10-4cf8-bd31-b9a02952acfb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59be342541264942ad87f60ff4de6c8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:14.152412 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8df5a53-2e10-4cf8-bd31-b9a02952acfb service cinder] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:14.152784 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8df5a53-2e10-4cf8-bd31-b9a02952acfb service cinder] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:14.153067 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8df5a53-2e10-4cf8-bd31-b9a02952acfb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:14.162660 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8df5a53-2e10-4cf8-bd31-b9a02952acfb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:14.164378 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8df5a53-2e10-4cf8-bd31-b9a02952acfb service cinder] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:14.175787 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1281/2546] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:51:14 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:51:14.226504 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7022dd8c-8f3a-42fa-9d71-bcf09885b27b None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:14.228638 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-7022dd8c-8f3a-42fa-9d71-bcf09885b27b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:14.241964 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7022dd8c-8f3a-42fa-9d71-bcf09885b27b None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:14.246005 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8b0afa-4201-4d34-b51a-df2b06a7c867 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59be342541264942ad87f60ff4de6c8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:14.246629 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8b0afa-4201-4d34-b51a-df2b06a7c867 service cinder] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:14.246907 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8b0afa-4201-4d34-b51a-df2b06a7c867 service cinder] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:14.247182 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8b0afa-4201-4d34-b51a-df2b06a7c867 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:14.315136 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8b0afa-4201-4d34-b51a-df2b06a7c867 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:14.316650 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8b0afa-4201-4d34-b51a-df2b06a7c867 service cinder] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:14.326993 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1266/2547] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:51:14 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:51:17.793441 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8df5a53-2e10-4cf8-bd31-b9a02952acfb service cinder] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:17.859259 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8df5a53-2e10-4cf8-bd31-b9a02952acfb service cinder] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:17.864158 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1738b7f-c81d-4ddf-a739-0154b3d02e43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:17.865476 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1738b7f-c81d-4ddf-a739-0154b3d02e43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:17.865758 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1738b7f-c81d-4ddf-a739-0154b3d02e43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:17.866028 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1738b7f-c81d-4ddf-a739-0154b3d02e43 admin admin] PATH_INFO: `/v3/users/eeb19c1e1d1148f4a0c5ac64393afec4` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:17.867184 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1738b7f-c81d-4ddf-a739-0154b3d02e43 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eeb19c1e1d1148f4a0c5ac64393afec4)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:17.869134 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1738b7f-c81d-4ddf-a739-0154b3d02e43 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:17.874476 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:51:17.920757 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-b1738b7f-c81d-4ddf-a739-0154b3d02e43 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eeb19c1e1d1148f4a0c5ac64393afec4'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:17.921388 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-b1738b7f-c81d-4ddf-a739-0154b3d02e43 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eeb19c1e1d1148f4a0c5ac64393afec4'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:17.928484 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-b1738b7f-c81d-4ddf-a739-0154b3d02e43 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eeb19c1e1d1148f4a0c5ac64393afec4'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:17.929228 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-b1738b7f-c81d-4ddf-a739-0154b3d02e43 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eeb19c1e1d1148f4a0c5ac64393afec4'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:17.942073 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-b1738b7f-c81d-4ddf-a739-0154b3d02e43 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eeb19c1e1d1148f4a0c5ac64393afec4'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:17.956046 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1282/2548] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 17:51:17 2022] DELETE /identity/v3/users/eeb19c1e1d1148f4a0c5ac64393afec4 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:51:17.961056 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2262dd-d3ff-447e-9eb7-04eb6999eaf1 None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:17.961533 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2262dd-d3ff-447e-9eb7-04eb6999eaf1 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:17.961646 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2262dd-d3ff-447e-9eb7-04eb6999eaf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:17.980697 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-4a2262dd-d3ff-447e-9eb7-04eb6999eaf1 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:51:17.981894 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-4a2262dd-d3ff-447e-9eb7-04eb6999eaf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:18.039314 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1267/2549] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:51:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:51:18.048885 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1738b7f-c81d-4ddf-a739-0154b3d02e43 admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:18.050267 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-b1738b7f-c81d-4ddf-a739-0154b3d02e43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:18.104980 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1738b7f-c81d-4ddf-a739-0154b3d02e43 admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:18.108538 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9690a8f4-9d9f-486e-a247-b7d57e85e3af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43ad98a250a144099ccd18378aedd4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:18.109157 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9690a8f4-9d9f-486e-a247-b7d57e85e3af service neutron] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:18.109488 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9690a8f4-9d9f-486e-a247-b7d57e85e3af service neutron] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:18.109702 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9690a8f4-9d9f-486e-a247-b7d57e85e3af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:18.118332 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9690a8f4-9d9f-486e-a247-b7d57e85e3af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:18.119750 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9690a8f4-9d9f-486e-a247-b7d57e85e3af service neutron] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:18.128920 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1283/2550] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:51:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:51:18.407135 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2262dd-d3ff-447e-9eb7-04eb6999eaf1 None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:18.408956 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-4a2262dd-d3ff-447e-9eb7-04eb6999eaf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:18.423315 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2262dd-d3ff-447e-9eb7-04eb6999eaf1 None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:18.428773 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cde566-2c32-43da-a682-c3267e5b5610 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:18.429050 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cde566-2c32-43da-a682-c3267e5b5610 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:18.429309 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cde566-2c32-43da-a682-c3267e5b5610 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:18.429677 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cde566-2c32-43da-a682-c3267e5b5610 admin admin] PATH_INFO: `/v3/projects/782d84bd35bf4a869c9054dcc87293cb` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:18.430942 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5cde566-2c32-43da-a682-c3267e5b5610 admin admin] RBAC: Authorizing `identity:delete_project(project_id=782d84bd35bf4a869c9054dcc87293cb)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:18.433100 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5cde566-2c32-43da-a682-c3267e5b5610 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:18.448675 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9690a8f4-9d9f-486e-a247-b7d57e85e3af service neutron] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:18.460416 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9690a8f4-9d9f-486e-a247-b7d57e85e3af service neutron] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:18.465671 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370eede7-c0d9-4920-a8fe-f8c8d1afc682 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43ad98a250a144099ccd18378aedd4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:18.466353 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370eede7-c0d9-4920-a8fe-f8c8d1afc682 service neutron] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:18.466702 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370eede7-c0d9-4920-a8fe-f8c8d1afc682 service neutron] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:18.467001 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370eede7-c0d9-4920-a8fe-f8c8d1afc682 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:18.468834 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-370eede7-c0d9-4920-a8fe-f8c8d1afc682 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:18.481942 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370eede7-c0d9-4920-a8fe-f8c8d1afc682 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:18.483368 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370eede7-c0d9-4920-a8fe-f8c8d1afc682 service neutron] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:18.489474 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-e5cde566-2c32-43da-a682-c3267e5b5610 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '782d84bd35bf4a869c9054dcc87293cb', 'request_id': 'req-e5cde566-2c32-43da-a682-c3267e5b5610'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:18.493559 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1284/2551] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:51:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 17:51:18.495077 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-e5cde566-2c32-43da-a682-c3267e5b5610 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '782d84bd35bf4a869c9054dcc87293cb', 'request_id': 'req-e5cde566-2c32-43da-a682-c3267e5b5610'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:18.501223 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-e5cde566-2c32-43da-a682-c3267e5b5610 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '782d84bd35bf4a869c9054dcc87293cb', 'request_id': 'req-e5cde566-2c32-43da-a682-c3267e5b5610'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:18.514986 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-e5cde566-2c32-43da-a682-c3267e5b5610 admin admin] The token cache is being invalidate because project 782d84bd35bf4a869c9054dcc87293cb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81759) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:51:18.515275 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-e5cde566-2c32-43da-a682-c3267e5b5610 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:18.516004 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-e5cde566-2c32-43da-a682-c3267e5b5610 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:18.517519 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1268/2552] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 17:51:18 2022] DELETE /identity/v3/projects/782d84bd35bf4a869c9054dcc87293cb => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:51:18.522229 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370eede7-c0d9-4920-a8fe-f8c8d1afc682 service neutron] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:18.531925 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370eede7-c0d9-4920-a8fe-f8c8d1afc682 service neutron] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:18.599632 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e8a759-efa0-4550-a155-beb704af795f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:18.599632 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e8a759-efa0-4550-a155-beb704af795f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:18.599632 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e8a759-efa0-4550-a155-beb704af795f admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:18.599632 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e8a759-efa0-4550-a155-beb704af795f admin admin] PATH_INFO: `/v3/users/7ce5dac03e3f48db96d697b33bef94ac` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:18.599962 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e8a759-efa0-4550-a155-beb704af795f admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ce5dac03e3f48db96d697b33bef94ac)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:18.601762 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e8a759-efa0-4550-a155-beb704af795f admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:18.621014 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:51:18.669364 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-99e8a759-efa0-4550-a155-beb704af795f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ce5dac03e3f48db96d697b33bef94ac'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:18.670253 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-99e8a759-efa0-4550-a155-beb704af795f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ce5dac03e3f48db96d697b33bef94ac'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:18.680704 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-99e8a759-efa0-4550-a155-beb704af795f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ce5dac03e3f48db96d697b33bef94ac'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:18.681424 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-99e8a759-efa0-4550-a155-beb704af795f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ce5dac03e3f48db96d697b33bef94ac'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:18.712012 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-99e8a759-efa0-4550-a155-beb704af795f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ce5dac03e3f48db96d697b33bef94ac'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:18.728590 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1285/2553] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 17:51:18 2022] DELETE /identity/v3/users/7ce5dac03e3f48db96d697b33bef94ac => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:51:19.158740 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cde566-2c32-43da-a682-c3267e5b5610 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:19.160343 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-e5cde566-2c32-43da-a682-c3267e5b5610 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:19.174011 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cde566-2c32-43da-a682-c3267e5b5610 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:19.183157 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:51:19.240184 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba1fe6f-cfbb-49af-ae17-7b12f46e09e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:19.240860 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba1fe6f-cfbb-49af-ae17-7b12f46e09e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:19.241140 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba1fe6f-cfbb-49af-ae17-7b12f46e09e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:19.241499 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba1fe6f-cfbb-49af-ae17-7b12f46e09e6 admin admin] PATH_INFO: `/v3/projects/c07a57edc321471eb53f70b54047d2f7` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:19.242547 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba1fe6f-cfbb-49af-ae17-7b12f46e09e6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c07a57edc321471eb53f70b54047d2f7)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:19.244468 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba1fe6f-cfbb-49af-ae17-7b12f46e09e6 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:19.293654 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-7ba1fe6f-cfbb-49af-ae17-7b12f46e09e6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c07a57edc321471eb53f70b54047d2f7', 'request_id': 'req-7ba1fe6f-cfbb-49af-ae17-7b12f46e09e6'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:19.301260 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-7ba1fe6f-cfbb-49af-ae17-7b12f46e09e6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c07a57edc321471eb53f70b54047d2f7', 'request_id': 'req-7ba1fe6f-cfbb-49af-ae17-7b12f46e09e6'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:19.308207 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-7ba1fe6f-cfbb-49af-ae17-7b12f46e09e6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c07a57edc321471eb53f70b54047d2f7', 'request_id': 'req-7ba1fe6f-cfbb-49af-ae17-7b12f46e09e6'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:19.323381 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-7ba1fe6f-cfbb-49af-ae17-7b12f46e09e6 admin admin] The token cache is being invalidate because project c07a57edc321471eb53f70b54047d2f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81759) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:51:19.323984 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-7ba1fe6f-cfbb-49af-ae17-7b12f46e09e6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:19.324749 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-7ba1fe6f-cfbb-49af-ae17-7b12f46e09e6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:19.326345 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1269/2554] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 17:51:19 2022] DELETE /identity/v3/projects/c07a57edc321471eb53f70b54047d2f7 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:51:19.332627 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0b6417-9e40-4da1-aa34-9060aebb59e1 None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:19.333200 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0b6417-9e40-4da1-aa34-9060aebb59e1 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:19.333833 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0b6417-9e40-4da1-aa34-9060aebb59e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:19.353973 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-ab0b6417-9e40-4da1-aa34-9060aebb59e1 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:51:19.355307 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-ab0b6417-9e40-4da1-aa34-9060aebb59e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:19.419077 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1286/2555] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:51:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:51:19.427712 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6391fdea-88bb-4146-ae9e-c2baa2f46245 None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:19.428000 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6391fdea-88bb-4146-ae9e-c2baa2f46245 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:19.428277 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6391fdea-88bb-4146-ae9e-c2baa2f46245 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:19.450766 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-6391fdea-88bb-4146-ae9e-c2baa2f46245 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:51:19.452008 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-6391fdea-88bb-4146-ae9e-c2baa2f46245 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:19.460204 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1270/2556] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:51:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:51:19.464616 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0b6417-9e40-4da1-aa34-9060aebb59e1 None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:19.474577 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0b6417-9e40-4da1-aa34-9060aebb59e1 None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:19.479188 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076e8ee3-dce9-4064-a112-a36c717f448c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:19.479797 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076e8ee3-dce9-4064-a112-a36c717f448c admin admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:19.480075 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076e8ee3-dce9-4064-a112-a36c717f448c admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:19.480350 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076e8ee3-dce9-4064-a112-a36c717f448c admin admin] PATH_INFO: `/v3/domains` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:19.481037 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076e8ee3-dce9-4064-a112-a36c717f448c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:51:19.481342 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076e8ee3-dce9-4064-a112-a36c717f448c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:19.482944 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076e8ee3-dce9-4064-a112-a36c717f448c admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:19.495673 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1287/2557] 10.222.0.45 () {58 vars in 1153 bytes} [Wed Aug 24 17:51:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:51:19.499883 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6391fdea-88bb-4146-ae9e-c2baa2f46245 None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:19.510347 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6391fdea-88bb-4146-ae9e-c2baa2f46245 None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:19.515183 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafa5512-bbbe-4174-a2c2-754324351b02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:19.515801 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafa5512-bbbe-4174-a2c2-754324351b02 admin admin] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:19.516079 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafa5512-bbbe-4174-a2c2-754324351b02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:19.516351 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafa5512-bbbe-4174-a2c2-754324351b02 admin admin] PATH_INFO: `/v3/projects` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:19.517042 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafa5512-bbbe-4174-a2c2-754324351b02 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:19.518982 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafa5512-bbbe-4174-a2c2-754324351b02 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:19.549545 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1271/2558] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 17:51:19 2022] POST /identity/v3/projects => generated 385 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:51:19.550635 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076e8ee3-dce9-4064-a112-a36c717f448c admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:19.561191 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076e8ee3-dce9-4064-a112-a36c717f448c admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:19.618933 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cf01f3-d5ec-4f26-b9dc-6905255c4ddf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:19.619553 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cf01f3-d5ec-4f26-b9dc-6905255c4ddf admin admin] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:19.619865 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cf01f3-d5ec-4f26-b9dc-6905255c4ddf admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:19.620119 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cf01f3-d5ec-4f26-b9dc-6905255c4ddf admin admin] PATH_INFO: `/v3/users` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:19.620789 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cf01f3-d5ec-4f26-b9dc-6905255c4ddf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:19.622721 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cf01f3-d5ec-4f26-b9dc-6905255c4ddf admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:19.644711 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1288/2559] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 17:51:19 2022] POST /identity/v3/users => generated 293 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:51:19.649004 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafa5512-bbbe-4174-a2c2-754324351b02 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:19.659341 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafa5512-bbbe-4174-a2c2-754324351b02 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:19.665175 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699dbcf5-e9a6-4f46-a9ac-e3e68068febc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:19.665783 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699dbcf5-e9a6-4f46-a9ac-e3e68068febc admin admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:19.666093 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699dbcf5-e9a6-4f46-a9ac-e3e68068febc admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:19.666370 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699dbcf5-e9a6-4f46-a9ac-e3e68068febc admin admin] PATH_INFO: `/v3/roles` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:19.667021 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699dbcf5-e9a6-4f46-a9ac-e3e68068febc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:19.668662 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699dbcf5-e9a6-4f46-a9ac-e3e68068febc admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:19.680065 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1272/2560] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:51:19 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:51:19.684099 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cf01f3-d5ec-4f26-b9dc-6905255c4ddf admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:19.694851 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cf01f3-d5ec-4f26-b9dc-6905255c4ddf admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:19.702965 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c27ccd-6ac8-4686-ba37-3d1c0eeee740 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:19.703597 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c27ccd-6ac8-4686-ba37-3d1c0eeee740 admin admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:19.704011 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c27ccd-6ac8-4686-ba37-3d1c0eeee740 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:19.704131 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c27ccd-6ac8-4686-ba37-3d1c0eeee740 admin admin] PATH_INFO: `/v3/roles` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:19.704851 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c27ccd-6ac8-4686-ba37-3d1c0eeee740 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:19.706480 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c27ccd-6ac8-4686-ba37-3d1c0eeee740 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:19.712795 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:51:19.721022 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1289/2561] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:51:19 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:51:19.723728 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699dbcf5-e9a6-4f46-a9ac-e3e68068febc admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:19.734127 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699dbcf5-e9a6-4f46-a9ac-e3e68068febc admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:19.738958 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cdb82d-d1e7-4129-ba50-9306e9a438d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:19.740128 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cdb82d-d1e7-4129-ba50-9306e9a438d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:19.740661 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cdb82d-d1e7-4129-ba50-9306e9a438d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:19.740790 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cdb82d-d1e7-4129-ba50-9306e9a438d9 admin admin] PATH_INFO: `/v3/projects/4eded6b57db7437f82fbcb0a5a7f5c2a/users/e9495aea97854ff3980d0b1e277b5617/roles/02476a5b82eb4b188c18a5d706a3ef68` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:19.769338 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cdb82d-d1e7-4129-ba50-9306e9a438d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4eded6b57db7437f82fbcb0a5a7f5c2a, user_id=e9495aea97854ff3980d0b1e277b5617, role_id=02476a5b82eb4b188c18a5d706a3ef68)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:19.772145 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cdb82d-d1e7-4129-ba50-9306e9a438d9 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:19.790109 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1273/2562] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 17:51:19 2022] PUT /identity/v3/projects/4eded6b57db7437f82fbcb0a5a7f5c2a/users/e9495aea97854ff3980d0b1e277b5617/roles/02476a5b82eb4b188c18a5d706a3ef68 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:51:19.802658 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1e27fe-b4da-4d6a-bd0f-0509fc26f8c2 None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:19.802923 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1e27fe-b4da-4d6a-bd0f-0509fc26f8c2 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:19.803359 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1e27fe-b4da-4d6a-bd0f-0509fc26f8c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:19.873894 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-9f1e27fe-b4da-4d6a-bd0f-0509fc26f8c2 None None] MFA Rules not processed for user `e9495aea97854ff3980d0b1e277b5617`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:51:19.875987 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-9f1e27fe-b4da-4d6a-bd0f-0509fc26f8c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:19.919058 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cdb82d-d1e7-4129-ba50-9306e9a438d9 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:19.921050 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-a7cdb82d-d1e7-4129-ba50-9306e9a438d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:19.967747 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1290/2563] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:51:19 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:51:19.969095 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:51:19.972187 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1e27fe-b4da-4d6a-bd0f-0509fc26f8c2 None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:19.991274 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cdb82d-d1e7-4129-ba50-9306e9a438d9 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:19.996502 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b70ae4-e307-45b6-9ffc-8e4d695355c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43ad98a250a144099ccd18378aedd4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:19.996896 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b70ae4-e307-45b6-9ffc-8e4d695355c1 service neutron] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:19.997171 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b70ae4-e307-45b6-9ffc-8e4d695355c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:19.997564 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b70ae4-e307-45b6-9ffc-8e4d695355c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:19.999783 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-57b70ae4-e307-45b6-9ffc-8e4d695355c1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:20.042231 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1e27fe-b4da-4d6a-bd0f-0509fc26f8c2 None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:20.048059 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee317af-de42-4e3a-aeb0-1fd366a593bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:20.048798 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee317af-de42-4e3a-aeb0-1fd366a593bb admin admin] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:20.049149 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee317af-de42-4e3a-aeb0-1fd366a593bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:20.049500 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee317af-de42-4e3a-aeb0-1fd366a593bb admin admin] PATH_INFO: `/v3/projects` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:20.050192 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee317af-de42-4e3a-aeb0-1fd366a593bb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:20.053364 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee317af-de42-4e3a-aeb0-1fd366a593bb admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:20.071509 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b70ae4-e307-45b6-9ffc-8e4d695355c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:20.073176 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b70ae4-e307-45b6-9ffc-8e4d695355c1 service neutron] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:20.083712 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1274/2564] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:51:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 17:51:20.087858 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1291/2565] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 17:51:19 2022] POST /identity/v3/projects => generated 383 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:51:20.089542 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b70ae4-e307-45b6-9ffc-8e4d695355c1 service neutron] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:20.100352 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b70ae4-e307-45b6-9ffc-8e4d695355c1 service neutron] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:20.178641 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec7f629-cb20-47e2-960d-963fd22cb963 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:20.179340 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec7f629-cb20-47e2-960d-963fd22cb963 admin admin] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:20.179537 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec7f629-cb20-47e2-960d-963fd22cb963 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:20.180218 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec7f629-cb20-47e2-960d-963fd22cb963 admin admin] PATH_INFO: `/v3/users` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:20.180427 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec7f629-cb20-47e2-960d-963fd22cb963 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:20.182415 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec7f629-cb20-47e2-960d-963fd22cb963 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:20.212811 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1275/2566] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 17:51:20 2022] POST /identity/v3/users => generated 298 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:51:20.213454 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee317af-de42-4e3a-aeb0-1fd366a593bb admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:20.229244 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee317af-de42-4e3a-aeb0-1fd366a593bb admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:20.235273 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd26807-eae1-4381-b998-a18b775184fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:20.235892 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd26807-eae1-4381-b998-a18b775184fa admin admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:20.236241 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd26807-eae1-4381-b998-a18b775184fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:20.236521 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd26807-eae1-4381-b998-a18b775184fa admin admin] PATH_INFO: `/v3/roles` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:20.243053 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd26807-eae1-4381-b998-a18b775184fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:20.244513 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd26807-eae1-4381-b998-a18b775184fa admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:20.262826 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1292/2567] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:51:20 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:51:20.263755 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec7f629-cb20-47e2-960d-963fd22cb963 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:20.276681 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec7f629-cb20-47e2-960d-963fd22cb963 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:20.281441 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56fecb5-ff7a-4688-b6ef-34911b6506ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:20.282224 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56fecb5-ff7a-4688-b6ef-34911b6506ab admin admin] REQUEST_METHOD: `PUT` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:20.282507 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56fecb5-ff7a-4688-b6ef-34911b6506ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:20.282781 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56fecb5-ff7a-4688-b6ef-34911b6506ab admin admin] PATH_INFO: `/v3/projects/f0d418521bde41f39217829e4b8ed11c/users/28f76d528864447f90425edf37ac6121/roles/e411ad9eb96e4e8d85b59b5b700ada7b` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:20.300690 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56fecb5-ff7a-4688-b6ef-34911b6506ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=f0d418521bde41f39217829e4b8ed11c, user_id=28f76d528864447f90425edf37ac6121, role_id=e411ad9eb96e4e8d85b59b5b700ada7b)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:20.303777 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56fecb5-ff7a-4688-b6ef-34911b6506ab admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:20.324045 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1276/2568] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 17:51:20 2022] PUT /identity/v3/projects/f0d418521bde41f39217829e4b8ed11c/users/28f76d528864447f90425edf37ac6121/roles/e411ad9eb96e4e8d85b59b5b700ada7b => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:51:20.325842 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd26807-eae1-4381-b998-a18b775184fa admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:20.338549 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd26807-eae1-4381-b998-a18b775184fa admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:20.426433 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841ef0bf-d423-4e61-a26f-737f01bd0673 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:20.427358 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841ef0bf-d423-4e61-a26f-737f01bd0673 admin admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:20.427642 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841ef0bf-d423-4e61-a26f-737f01bd0673 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:20.427954 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841ef0bf-d423-4e61-a26f-737f01bd0673 admin admin] PATH_INFO: `/v3/roles` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:20.428633 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841ef0bf-d423-4e61-a26f-737f01bd0673 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:20.430303 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841ef0bf-d423-4e61-a26f-737f01bd0673 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:20.433121 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:51:20.444475 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1293/2569] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:51:20 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:51:20.448505 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56fecb5-ff7a-4688-b6ef-34911b6506ab admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:20.463340 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56fecb5-ff7a-4688-b6ef-34911b6506ab admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:20.469641 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e9e9c3-d7d3-448e-837d-88cd74b11386 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:20.471971 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e9e9c3-d7d3-448e-837d-88cd74b11386 admin admin] REQUEST_METHOD: `PUT` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:20.472421 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e9e9c3-d7d3-448e-837d-88cd74b11386 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:20.472890 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e9e9c3-d7d3-448e-837d-88cd74b11386 admin admin] PATH_INFO: `/v3/projects/f0d418521bde41f39217829e4b8ed11c/users/28f76d528864447f90425edf37ac6121/roles/e411ad9eb96e4e8d85b59b5b700ada7b` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:20.475667 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e9e9c3-d7d3-448e-837d-88cd74b11386 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f0d418521bde41f39217829e4b8ed11c, user_id=28f76d528864447f90425edf37ac6121, role_id=e411ad9eb96e4e8d85b59b5b700ada7b)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:20.478889 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e9e9c3-d7d3-448e-837d-88cd74b11386 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:20.492160 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1277/2570] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 17:51:20 2022] PUT /identity/v3/projects/f0d418521bde41f39217829e4b8ed11c/users/28f76d528864447f90425edf37ac6121/roles/e411ad9eb96e4e8d85b59b5b700ada7b => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:51:20.502133 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33045d5-1e49-4a7d-872b-8e1a592f49bf None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:20.502440 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33045d5-1e49-4a7d-872b-8e1a592f49bf None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:20.502702 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33045d5-1e49-4a7d-872b-8e1a592f49bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:20.543072 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-a33045d5-1e49-4a7d-872b-8e1a592f49bf None None] MFA Rules not processed for user `28f76d528864447f90425edf37ac6121`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:51:20.544930 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-a33045d5-1e49-4a7d-872b-8e1a592f49bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:20.625463 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1294/2571] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:51:20 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:51:20.636884 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e9e9c3-d7d3-448e-837d-88cd74b11386 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:20.639044 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-08e9e9c3-d7d3-448e-837d-88cd74b11386 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:20.699505 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:51:20.724623 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e9e9c3-d7d3-448e-837d-88cd74b11386 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:20.728833 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d2dac8-cffb-40fd-bde8-9fb259d28c4e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59be342541264942ad87f60ff4de6c8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:20.729475 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d2dac8-cffb-40fd-bde8-9fb259d28c4e service cinder] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:20.729755 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d2dac8-cffb-40fd-bde8-9fb259d28c4e service cinder] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:20.730028 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d2dac8-cffb-40fd-bde8-9fb259d28c4e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:20.821450 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d2dac8-cffb-40fd-bde8-9fb259d28c4e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:20.822935 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d2dac8-cffb-40fd-bde8-9fb259d28c4e service cinder] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:20.833168 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1278/2572] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:51:20 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:51:24.913805 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33045d5-1e49-4a7d-872b-8e1a592f49bf None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:24.915517 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-a33045d5-1e49-4a7d-872b-8e1a592f49bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:24.982189 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33045d5-1e49-4a7d-872b-8e1a592f49bf None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:24.987560 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d5fd7b-bdeb-4f15-b9ad-745ef03f8fd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:24.988199 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d5fd7b-bdeb-4f15-b9ad-745ef03f8fd3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:24.988479 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d5fd7b-bdeb-4f15-b9ad-745ef03f8fd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:24.988786 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d5fd7b-bdeb-4f15-b9ad-745ef03f8fd3 admin admin] PATH_INFO: `/v3/users/892dad01ea684e6cb2d5cec659c6c4b9` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:25.004221 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d5fd7b-bdeb-4f15-b9ad-745ef03f8fd3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=892dad01ea684e6cb2d5cec659c6c4b9)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:25.006882 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d5fd7b-bdeb-4f15-b9ad-745ef03f8fd3 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:25.059867 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-f7d5fd7b-bdeb-4f15-b9ad-745ef03f8fd3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '892dad01ea684e6cb2d5cec659c6c4b9'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:25.060628 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-f7d5fd7b-bdeb-4f15-b9ad-745ef03f8fd3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '892dad01ea684e6cb2d5cec659c6c4b9'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:25.068811 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-f7d5fd7b-bdeb-4f15-b9ad-745ef03f8fd3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '892dad01ea684e6cb2d5cec659c6c4b9'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:25.069538 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-f7d5fd7b-bdeb-4f15-b9ad-745ef03f8fd3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '892dad01ea684e6cb2d5cec659c6c4b9'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:25.083392 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-f7d5fd7b-bdeb-4f15-b9ad-745ef03f8fd3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '892dad01ea684e6cb2d5cec659c6c4b9'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:25.085293 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:51:25.101133 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1295/2573] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 17:51:24 2022] DELETE /identity/v3/users/892dad01ea684e6cb2d5cec659c6c4b9 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:51:25.303564 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d2dac8-cffb-40fd-bde8-9fb259d28c4e service cinder] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:25.326579 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d2dac8-cffb-40fd-bde8-9fb259d28c4e service cinder] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:25.414358 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4de285-74a9-4e74-947b-616209a00dd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:25.414997 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4de285-74a9-4e74-947b-616209a00dd7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:25.415276 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4de285-74a9-4e74-947b-616209a00dd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:25.415616 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4de285-74a9-4e74-947b-616209a00dd7 admin admin] PATH_INFO: `/v3/projects/761d3524ec174bcb8f207cff3728bcea` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:25.428402 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4de285-74a9-4e74-947b-616209a00dd7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=761d3524ec174bcb8f207cff3728bcea)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:25.430503 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4de285-74a9-4e74-947b-616209a00dd7 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:25.454094 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d5fd7b-bdeb-4f15-b9ad-745ef03f8fd3 admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:25.455831 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-f7d5fd7b-bdeb-4f15-b9ad-745ef03f8fd3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:25.490289 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-1d4de285-74a9-4e74-947b-616209a00dd7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '761d3524ec174bcb8f207cff3728bcea', 'request_id': 'req-1d4de285-74a9-4e74-947b-616209a00dd7'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:25.496428 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-1d4de285-74a9-4e74-947b-616209a00dd7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '761d3524ec174bcb8f207cff3728bcea', 'request_id': 'req-1d4de285-74a9-4e74-947b-616209a00dd7'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:25.504763 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-1d4de285-74a9-4e74-947b-616209a00dd7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '761d3524ec174bcb8f207cff3728bcea', 'request_id': 'req-1d4de285-74a9-4e74-947b-616209a00dd7'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:25.522489 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-1d4de285-74a9-4e74-947b-616209a00dd7 admin admin] The token cache is being invalidate because project 761d3524ec174bcb8f207cff3728bcea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81759) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:51:25.522939 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-1d4de285-74a9-4e74-947b-616209a00dd7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:25.523698 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-1d4de285-74a9-4e74-947b-616209a00dd7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:25.525759 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1279/2574] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 17:51:25 2022] DELETE /identity/v3/projects/761d3524ec174bcb8f207cff3728bcea => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:51:25.531555 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9702982-5977-44fc-af0e-77f5ddc7ba36 None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:25.531945 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9702982-5977-44fc-af0e-77f5ddc7ba36 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:25.532848 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9702982-5977-44fc-af0e-77f5ddc7ba36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:25.534173 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d5fd7b-bdeb-4f15-b9ad-745ef03f8fd3 admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:25.568694 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-a9702982-5977-44fc-af0e-77f5ddc7ba36 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:51:25.570747 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-a9702982-5977-44fc-af0e-77f5ddc7ba36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:25.582264 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:51:25.600743 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74f851a-3a0e-4cec-940e-fb24664d5085 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59be342541264942ad87f60ff4de6c8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:25.601738 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74f851a-3a0e-4cec-940e-fb24664d5085 service cinder] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:25.602686 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74f851a-3a0e-4cec-940e-fb24664d5085 service cinder] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:25.603159 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74f851a-3a0e-4cec-940e-fb24664d5085 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:25.605716 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-f74f851a-3a0e-4cec-940e-fb24664d5085 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:25.662836 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1280/2575] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:51:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:51:25.674858 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d657f612-0dd6-4ca8-a0c2-1d2f0e06610b None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:25.675152 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d657f612-0dd6-4ca8-a0c2-1d2f0e06610b None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:25.675481 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d657f612-0dd6-4ca8-a0c2-1d2f0e06610b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:25.703496 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74f851a-3a0e-4cec-940e-fb24664d5085 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:25.705905 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74f851a-3a0e-4cec-940e-fb24664d5085 service cinder] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:25.707278 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-d657f612-0dd6-4ca8-a0c2-1d2f0e06610b None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:51:25.709085 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-d657f612-0dd6-4ca8-a0c2-1d2f0e06610b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:25.721099 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1281/2576] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:51:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:51:25.725639 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d657f612-0dd6-4ca8-a0c2-1d2f0e06610b None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:25.729688 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1296/2577] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:51:25 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 276 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 17:51:25.735678 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d657f612-0dd6-4ca8-a0c2-1d2f0e06610b None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:25.740927 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52966a1a-7698-47a2-9ba9-e7aef86f1fcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:25.741577 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52966a1a-7698-47a2-9ba9-e7aef86f1fcd admin admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:25.741860 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52966a1a-7698-47a2-9ba9-e7aef86f1fcd admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:25.742130 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52966a1a-7698-47a2-9ba9-e7aef86f1fcd admin admin] PATH_INFO: `/v3/domains` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:25.742777 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52966a1a-7698-47a2-9ba9-e7aef86f1fcd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:51:25.743079 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52966a1a-7698-47a2-9ba9-e7aef86f1fcd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:25.744699 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52966a1a-7698-47a2-9ba9-e7aef86f1fcd admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:25.763415 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1282/2578] 10.222.0.45 () {58 vars in 1153 bytes} [Wed Aug 24 17:51:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:51:25.768404 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74f851a-3a0e-4cec-940e-fb24664d5085 service cinder] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:25.787849 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74f851a-3a0e-4cec-940e-fb24664d5085 service cinder] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:25.795241 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f027b59d-5e4c-4c99-8841-8e03da73a279 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:25.795939 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f027b59d-5e4c-4c99-8841-8e03da73a279 admin admin] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:25.796359 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f027b59d-5e4c-4c99-8841-8e03da73a279 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:25.796763 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f027b59d-5e4c-4c99-8841-8e03da73a279 admin admin] PATH_INFO: `/v3/projects` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:25.797613 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f027b59d-5e4c-4c99-8841-8e03da73a279 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:25.800871 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f027b59d-5e4c-4c99-8841-8e03da73a279 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:25.836606 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1297/2579] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 17:51:25 2022] POST /identity/v3/projects => generated 371 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:51:25.838471 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52966a1a-7698-47a2-9ba9-e7aef86f1fcd admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:25.850805 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52966a1a-7698-47a2-9ba9-e7aef86f1fcd admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:25.918850 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac27abd-c532-42be-b834-ddfc79aa4933 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:25.919468 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac27abd-c532-42be-b834-ddfc79aa4933 admin admin] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:25.919754 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac27abd-c532-42be-b834-ddfc79aa4933 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:25.920019 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac27abd-c532-42be-b834-ddfc79aa4933 admin admin] PATH_INFO: `/v3/users` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:25.920682 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac27abd-c532-42be-b834-ddfc79aa4933 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:25.922794 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac27abd-c532-42be-b834-ddfc79aa4933 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:25.954709 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1283/2580] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 17:51:25 2022] POST /identity/v3/users => generated 286 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:51:25.955258 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f027b59d-5e4c-4c99-8841-8e03da73a279 admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:25.968775 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f027b59d-5e4c-4c99-8841-8e03da73a279 admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:25.986413 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ade1a3-4b82-4ca7-a112-df1f42f8268e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:25.988529 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ade1a3-4b82-4ca7-a112-df1f42f8268e admin admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:25.988797 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ade1a3-4b82-4ca7-a112-df1f42f8268e admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:25.989054 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ade1a3-4b82-4ca7-a112-df1f42f8268e admin admin] PATH_INFO: `/v3/roles` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:25.991178 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ade1a3-4b82-4ca7-a112-df1f42f8268e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:26.004305 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ade1a3-4b82-4ca7-a112-df1f42f8268e admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:26.021097 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1298/2581] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:51:25 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:51:26.022695 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac27abd-c532-42be-b834-ddfc79aa4933 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:26.036099 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac27abd-c532-42be-b834-ddfc79aa4933 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:26.050339 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b3f206-a832-44ad-8616-b26adfb6f8b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:26.051480 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b3f206-a832-44ad-8616-b26adfb6f8b3 admin admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:26.051480 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b3f206-a832-44ad-8616-b26adfb6f8b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:26.051699 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b3f206-a832-44ad-8616-b26adfb6f8b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:26.057227 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b3f206-a832-44ad-8616-b26adfb6f8b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:26.058886 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b3f206-a832-44ad-8616-b26adfb6f8b3 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:26.071108 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1284/2582] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:51:26 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:51:26.075658 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ade1a3-4b82-4ca7-a112-df1f42f8268e admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:26.087326 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ade1a3-4b82-4ca7-a112-df1f42f8268e admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:26.093879 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854c55bc-8eaf-4f4e-b57d-49fbe0d55b49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:26.108500 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854c55bc-8eaf-4f4e-b57d-49fbe0d55b49 admin admin] REQUEST_METHOD: `PUT` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:26.108500 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854c55bc-8eaf-4f4e-b57d-49fbe0d55b49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:26.108500 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854c55bc-8eaf-4f4e-b57d-49fbe0d55b49 admin admin] PATH_INFO: `/v3/projects/88f97da619354ec781766f418079da18/users/56d7d40ab72d459ab4ae8fa5bb245a6d/roles/02476a5b82eb4b188c18a5d706a3ef68` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:26.122646 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854c55bc-8eaf-4f4e-b57d-49fbe0d55b49 admin admin] RBAC: Authorizing `identity:create_grant(project_id=88f97da619354ec781766f418079da18, user_id=56d7d40ab72d459ab4ae8fa5bb245a6d, role_id=02476a5b82eb4b188c18a5d706a3ef68)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:26.126071 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854c55bc-8eaf-4f4e-b57d-49fbe0d55b49 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:26.142565 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:INSERT=3,keystone:DELETE=1,keystone:SELECT=96 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:51:26.149646 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1299/2583] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 17:51:26 2022] PUT /identity/v3/projects/88f97da619354ec781766f418079da18/users/56d7d40ab72d459ab4ae8fa5bb245a6d/roles/02476a5b82eb4b188c18a5d706a3ef68 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:51:26.164421 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60309b9-1a67-428e-b591-6ea5b467e027 None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:26.164797 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60309b9-1a67-428e-b591-6ea5b467e027 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:26.165050 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60309b9-1a67-428e-b591-6ea5b467e027 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:26.218798 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-a60309b9-1a67-428e-b591-6ea5b467e027 None None] MFA Rules not processed for user `56d7d40ab72d459ab4ae8fa5bb245a6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:51:26.221827 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-a60309b9-1a67-428e-b591-6ea5b467e027 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:26.300972 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1285/2584] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:51:26 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:51:26.312704 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854c55bc-8eaf-4f4e-b57d-49fbe0d55b49 admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:26.314546 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-854c55bc-8eaf-4f4e-b57d-49fbe0d55b49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:26.399592 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854c55bc-8eaf-4f4e-b57d-49fbe0d55b49 admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:26.405320 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc40a188-f0ec-404b-ac5c-1a7cfd2ac21e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '053b7ce6d6f649ed91a26e434dab805e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:26.405945 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc40a188-f0ec-404b-ac5c-1a7cfd2ac21e service nova] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:26.406230 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc40a188-f0ec-404b-ac5c-1a7cfd2ac21e service nova] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:26.406504 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc40a188-f0ec-404b-ac5c-1a7cfd2ac21e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:26.416705 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc40a188-f0ec-404b-ac5c-1a7cfd2ac21e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:26.418380 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc40a188-f0ec-404b-ac5c-1a7cfd2ac21e service nova] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:26.428779 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1300/2585] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:51:26 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:51:27.277015 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60309b9-1a67-428e-b591-6ea5b467e027 None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:27.279724 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-a60309b9-1a67-428e-b591-6ea5b467e027 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:27.287390 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:51:27.360411 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60309b9-1a67-428e-b591-6ea5b467e027 None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:27.366490 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5caf3e1-8ef1-4fe6-846c-a4cdad2d4f2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:27.367136 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5caf3e1-8ef1-4fe6-846c-a4cdad2d4f2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:27.367419 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5caf3e1-8ef1-4fe6-846c-a4cdad2d4f2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:27.367744 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5caf3e1-8ef1-4fe6-846c-a4cdad2d4f2c admin admin] PATH_INFO: `/v3/users/af6710bfe62d4ba29dc48e2f7aba9ec5` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:27.384575 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5caf3e1-8ef1-4fe6-846c-a4cdad2d4f2c admin admin] RBAC: Authorizing `identity:delete_user(user_id=af6710bfe62d4ba29dc48e2f7aba9ec5)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:27.386616 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5caf3e1-8ef1-4fe6-846c-a4cdad2d4f2c admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:27.442197 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-a5caf3e1-8ef1-4fe6-846c-a4cdad2d4f2c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af6710bfe62d4ba29dc48e2f7aba9ec5'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:27.443084 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-a5caf3e1-8ef1-4fe6-846c-a4cdad2d4f2c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af6710bfe62d4ba29dc48e2f7aba9ec5'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:27.454057 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-a5caf3e1-8ef1-4fe6-846c-a4cdad2d4f2c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af6710bfe62d4ba29dc48e2f7aba9ec5'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:27.454804 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-a5caf3e1-8ef1-4fe6-846c-a4cdad2d4f2c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af6710bfe62d4ba29dc48e2f7aba9ec5'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:27.462169 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-a5caf3e1-8ef1-4fe6-846c-a4cdad2d4f2c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af6710bfe62d4ba29dc48e2f7aba9ec5'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:27.480830 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1286/2586] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 17:51:27 2022] DELETE /identity/v3/users/af6710bfe62d4ba29dc48e2f7aba9ec5 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:51:27.668946 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc40a188-f0ec-404b-ac5c-1a7cfd2ac21e service nova] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:27.686460 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc40a188-f0ec-404b-ac5c-1a7cfd2ac21e service nova] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:27.747335 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdae9f01-c5a2-4e7f-8ba2-750cefe5edb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:27.747969 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdae9f01-c5a2-4e7f-8ba2-750cefe5edb0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:27.748248 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdae9f01-c5a2-4e7f-8ba2-750cefe5edb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:27.748521 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdae9f01-c5a2-4e7f-8ba2-750cefe5edb0 admin admin] PATH_INFO: `/v3/projects/87a610c98b9b48bd96d2954604bf3078` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:27.758981 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdae9f01-c5a2-4e7f-8ba2-750cefe5edb0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=87a610c98b9b48bd96d2954604bf3078)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:27.760977 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdae9f01-c5a2-4e7f-8ba2-750cefe5edb0 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:27.815538 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-fdae9f01-c5a2-4e7f-8ba2-750cefe5edb0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '87a610c98b9b48bd96d2954604bf3078', 'request_id': 'req-fdae9f01-c5a2-4e7f-8ba2-750cefe5edb0'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:27.822603 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-fdae9f01-c5a2-4e7f-8ba2-750cefe5edb0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '87a610c98b9b48bd96d2954604bf3078', 'request_id': 'req-fdae9f01-c5a2-4e7f-8ba2-750cefe5edb0'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:27.829502 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-fdae9f01-c5a2-4e7f-8ba2-750cefe5edb0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '87a610c98b9b48bd96d2954604bf3078', 'request_id': 'req-fdae9f01-c5a2-4e7f-8ba2-750cefe5edb0'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:27.840662 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-fdae9f01-c5a2-4e7f-8ba2-750cefe5edb0 admin admin] The token cache is being invalidate because project 87a610c98b9b48bd96d2954604bf3078 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81758) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:51:27.840953 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-fdae9f01-c5a2-4e7f-8ba2-750cefe5edb0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:27.841599 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-fdae9f01-c5a2-4e7f-8ba2-750cefe5edb0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:27.843080 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1301/2587] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 17:51:27 2022] DELETE /identity/v3/projects/87a610c98b9b48bd96d2954604bf3078 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:51:27.847090 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5caf3e1-8ef1-4fe6-846c-a4cdad2d4f2c admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:27.848606 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-a5caf3e1-8ef1-4fe6-846c-a4cdad2d4f2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:27.863271 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5caf3e1-8ef1-4fe6-846c-a4cdad2d4f2c admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:27.919607 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad92c870-c8fc-4be0-a7d2-a129cfebce70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:27.920158 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad92c870-c8fc-4be0-a7d2-a129cfebce70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:27.920441 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad92c870-c8fc-4be0-a7d2-a129cfebce70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:27.920762 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad92c870-c8fc-4be0-a7d2-a129cfebce70 admin admin] PATH_INFO: `/v3/users/3c60d052608a4dce99944f07912a36f2` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:27.934013 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad92c870-c8fc-4be0-a7d2-a129cfebce70 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3c60d052608a4dce99944f07912a36f2)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:27.935954 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad92c870-c8fc-4be0-a7d2-a129cfebce70 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:27.938219 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:51:27.983024 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-ad92c870-c8fc-4be0-a7d2-a129cfebce70 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3c60d052608a4dce99944f07912a36f2'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:27.983654 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-ad92c870-c8fc-4be0-a7d2-a129cfebce70 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3c60d052608a4dce99944f07912a36f2'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:27.990457 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-ad92c870-c8fc-4be0-a7d2-a129cfebce70 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3c60d052608a4dce99944f07912a36f2'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:27.991075 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-ad92c870-c8fc-4be0-a7d2-a129cfebce70 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3c60d052608a4dce99944f07912a36f2'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:27.998260 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-ad92c870-c8fc-4be0-a7d2-a129cfebce70 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3c60d052608a4dce99944f07912a36f2'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:28.015581 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1287/2588] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 17:51:27 2022] DELETE /identity/v3/users/3c60d052608a4dce99944f07912a36f2 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:51:28.194541 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdae9f01-c5a2-4e7f-8ba2-750cefe5edb0 admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:28.196383 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-fdae9f01-c5a2-4e7f-8ba2-750cefe5edb0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:28.234379 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdae9f01-c5a2-4e7f-8ba2-750cefe5edb0 admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:28.239755 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:51:28.316747 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fc4410-3551-4ddc-9ff4-eed42af14946 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:28.318077 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fc4410-3551-4ddc-9ff4-eed42af14946 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:28.318718 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fc4410-3551-4ddc-9ff4-eed42af14946 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:28.318718 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fc4410-3551-4ddc-9ff4-eed42af14946 admin admin] PATH_INFO: `/v3/projects/a8040afb734748a4ac1698d26f7321fa` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:28.330893 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fc4410-3551-4ddc-9ff4-eed42af14946 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a8040afb734748a4ac1698d26f7321fa)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:28.333853 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fc4410-3551-4ddc-9ff4-eed42af14946 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:28.385667 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-a8fc4410-3551-4ddc-9ff4-eed42af14946 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a8040afb734748a4ac1698d26f7321fa', 'request_id': 'req-a8fc4410-3551-4ddc-9ff4-eed42af14946'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:28.394253 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-a8fc4410-3551-4ddc-9ff4-eed42af14946 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a8040afb734748a4ac1698d26f7321fa', 'request_id': 'req-a8fc4410-3551-4ddc-9ff4-eed42af14946'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:28.402038 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-a8fc4410-3551-4ddc-9ff4-eed42af14946 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a8040afb734748a4ac1698d26f7321fa', 'request_id': 'req-a8fc4410-3551-4ddc-9ff4-eed42af14946'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:28.415814 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-a8fc4410-3551-4ddc-9ff4-eed42af14946 admin admin] The token cache is being invalidate because project a8040afb734748a4ac1698d26f7321fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81758) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:51:28.416177 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-a8fc4410-3551-4ddc-9ff4-eed42af14946 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:28.417346 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-a8fc4410-3551-4ddc-9ff4-eed42af14946 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:28.418879 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1302/2589] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 17:51:28 2022] DELETE /identity/v3/projects/a8040afb734748a4ac1698d26f7321fa => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:51:29.335040 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad92c870-c8fc-4be0-a7d2-a129cfebce70 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:29.335040 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-ad92c870-c8fc-4be0-a7d2-a129cfebce70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:29.347450 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad92c870-c8fc-4be0-a7d2-a129cfebce70 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:29.402962 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fb1624-d691-4a09-b043-6544457b0ed7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:29.403585 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fb1624-d691-4a09-b043-6544457b0ed7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:29.403861 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fb1624-d691-4a09-b043-6544457b0ed7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:29.404132 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fb1624-d691-4a09-b043-6544457b0ed7 admin admin] PATH_INFO: `/v3/users/56d7d40ab72d459ab4ae8fa5bb245a6d` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:29.405251 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fb1624-d691-4a09-b043-6544457b0ed7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=56d7d40ab72d459ab4ae8fa5bb245a6d)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:29.407158 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fb1624-d691-4a09-b043-6544457b0ed7 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:29.452691 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-43fb1624-d691-4a09-b043-6544457b0ed7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '56d7d40ab72d459ab4ae8fa5bb245a6d'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:29.453369 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-43fb1624-d691-4a09-b043-6544457b0ed7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '56d7d40ab72d459ab4ae8fa5bb245a6d'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:29.460797 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-43fb1624-d691-4a09-b043-6544457b0ed7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '56d7d40ab72d459ab4ae8fa5bb245a6d'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:29.461545 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-43fb1624-d691-4a09-b043-6544457b0ed7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '56d7d40ab72d459ab4ae8fa5bb245a6d'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:29.469272 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-43fb1624-d691-4a09-b043-6544457b0ed7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '56d7d40ab72d459ab4ae8fa5bb245a6d'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:29.483872 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:51:29.488401 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1288/2590] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 17:51:29 2022] DELETE /identity/v3/users/56d7d40ab72d459ab4ae8fa5bb245a6d => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:51:29.493451 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54d2b0e-08d5-40a5-a780-45d166e85700 None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:29.493950 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54d2b0e-08d5-40a5-a780-45d166e85700 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:29.494046 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54d2b0e-08d5-40a5-a780-45d166e85700 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:29.512815 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-f54d2b0e-08d5-40a5-a780-45d166e85700 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:51:29.514091 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-f54d2b0e-08d5-40a5-a780-45d166e85700 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:29.574818 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1303/2591] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:51:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:51:29.584842 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fb1624-d691-4a09-b043-6544457b0ed7 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:29.586192 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-43fb1624-d691-4a09-b043-6544457b0ed7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:29.643346 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fb1624-d691-4a09-b043-6544457b0ed7 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:29.647732 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196d0af4-9cc1-4172-9044-46969e9192bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43ad98a250a144099ccd18378aedd4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:29.648286 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196d0af4-9cc1-4172-9044-46969e9192bd service neutron] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:29.648590 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196d0af4-9cc1-4172-9044-46969e9192bd service neutron] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:29.648871 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196d0af4-9cc1-4172-9044-46969e9192bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:29.657925 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196d0af4-9cc1-4172-9044-46969e9192bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:29.659365 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196d0af4-9cc1-4172-9044-46969e9192bd service neutron] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:29.669552 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1289/2592] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:51:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 17:51:29.946667 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54d2b0e-08d5-40a5-a780-45d166e85700 None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:29.948236 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-f54d2b0e-08d5-40a5-a780-45d166e85700 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:29.960176 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54d2b0e-08d5-40a5-a780-45d166e85700 None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:29.964729 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c090137c-ba26-4fa6-b5f4-e361aaa75035 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:29.965361 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c090137c-ba26-4fa6-b5f4-e361aaa75035 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:29.965638 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c090137c-ba26-4fa6-b5f4-e361aaa75035 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:29.965912 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c090137c-ba26-4fa6-b5f4-e361aaa75035 admin admin] PATH_INFO: `/v3/projects/88f97da619354ec781766f418079da18` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:29.967413 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c090137c-ba26-4fa6-b5f4-e361aaa75035 admin admin] RBAC: Authorizing `identity:delete_project(project_id=88f97da619354ec781766f418079da18)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:29.969351 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c090137c-ba26-4fa6-b5f4-e361aaa75035 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:29.993806 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:51:30.016539 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-c090137c-ba26-4fa6-b5f4-e361aaa75035 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '88f97da619354ec781766f418079da18', 'request_id': 'req-c090137c-ba26-4fa6-b5f4-e361aaa75035'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:30.023746 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-c090137c-ba26-4fa6-b5f4-e361aaa75035 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '88f97da619354ec781766f418079da18', 'request_id': 'req-c090137c-ba26-4fa6-b5f4-e361aaa75035'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:30.029744 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-c090137c-ba26-4fa6-b5f4-e361aaa75035 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '88f97da619354ec781766f418079da18', 'request_id': 'req-c090137c-ba26-4fa6-b5f4-e361aaa75035'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:30.039976 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-c090137c-ba26-4fa6-b5f4-e361aaa75035 admin admin] The token cache is being invalidate because project 88f97da619354ec781766f418079da18 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81758) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:51:30.040261 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-c090137c-ba26-4fa6-b5f4-e361aaa75035 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:30.040899 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-c090137c-ba26-4fa6-b5f4-e361aaa75035 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:30.042376 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1304/2593] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 17:51:29 2022] DELETE /identity/v3/projects/88f97da619354ec781766f418079da18 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:51:30.047652 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2970c6-fbc8-442f-871d-b4671b83f5da None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:30.048002 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2970c6-fbc8-442f-871d-b4671b83f5da None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:30.048354 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2970c6-fbc8-442f-871d-b4671b83f5da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:30.068865 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-0d2970c6-fbc8-442f-871d-b4671b83f5da None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:51:30.070059 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-0d2970c6-fbc8-442f-871d-b4671b83f5da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:30.127980 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1290/2594] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:51:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:51:30.137341 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1cc2a8-db5c-4822-b2b9-2707e501b8da None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:30.137629 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1cc2a8-db5c-4822-b2b9-2707e501b8da None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:30.137905 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1cc2a8-db5c-4822-b2b9-2707e501b8da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:30.157253 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-8e1cc2a8-db5c-4822-b2b9-2707e501b8da None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:51:30.158496 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-8e1cc2a8-db5c-4822-b2b9-2707e501b8da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:30.166651 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1305/2595] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:51:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:51:30.170702 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2970c6-fbc8-442f-871d-b4671b83f5da None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:30.180685 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2970c6-fbc8-442f-871d-b4671b83f5da None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:30.185284 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277525e5-767d-4ee3-a696-d402c94c7619 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:30.185877 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277525e5-767d-4ee3-a696-d402c94c7619 admin admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:30.186159 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277525e5-767d-4ee3-a696-d402c94c7619 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:30.186430 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277525e5-767d-4ee3-a696-d402c94c7619 admin admin] PATH_INFO: `/v3/domains` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:30.187068 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277525e5-767d-4ee3-a696-d402c94c7619 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:51:30.187396 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277525e5-767d-4ee3-a696-d402c94c7619 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:30.189006 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277525e5-767d-4ee3-a696-d402c94c7619 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:30.200674 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1291/2596] 10.222.0.45 () {58 vars in 1153 bytes} [Wed Aug 24 17:51:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:51:30.204544 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1cc2a8-db5c-4822-b2b9-2707e501b8da None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:30.213655 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1cc2a8-db5c-4822-b2b9-2707e501b8da None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:30.218318 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd07384-1d81-40fa-97a2-cd4828bafa8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:30.218905 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd07384-1d81-40fa-97a2-cd4828bafa8c admin admin] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:30.219211 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd07384-1d81-40fa-97a2-cd4828bafa8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:30.219508 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd07384-1d81-40fa-97a2-cd4828bafa8c admin admin] PATH_INFO: `/v3/projects` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:30.220138 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd07384-1d81-40fa-97a2-cd4828bafa8c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:30.222062 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd07384-1d81-40fa-97a2-cd4828bafa8c admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:30.248623 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1306/2597] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 17:51:30 2022] POST /identity/v3/projects => generated 369 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:51:30.253340 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277525e5-767d-4ee3-a696-d402c94c7619 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:30.270826 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277525e5-767d-4ee3-a696-d402c94c7619 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:30.337085 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74caab5-87c1-4492-9b17-19c77bdcc869 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:30.337845 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74caab5-87c1-4492-9b17-19c77bdcc869 admin admin] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:30.337977 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74caab5-87c1-4492-9b17-19c77bdcc869 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:30.338217 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74caab5-87c1-4492-9b17-19c77bdcc869 admin admin] PATH_INFO: `/v3/users` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:30.338835 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74caab5-87c1-4492-9b17-19c77bdcc869 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:30.340816 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74caab5-87c1-4492-9b17-19c77bdcc869 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:30.361578 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1292/2598] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 17:51:30 2022] POST /identity/v3/users => generated 285 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:51:30.366246 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd07384-1d81-40fa-97a2-cd4828bafa8c admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:30.376492 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd07384-1d81-40fa-97a2-cd4828bafa8c admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:30.381151 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e0e30f-4b55-461b-8a44-fdc0d4ecf8f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:30.381688 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e0e30f-4b55-461b-8a44-fdc0d4ecf8f3 admin admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:30.381968 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e0e30f-4b55-461b-8a44-fdc0d4ecf8f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:30.382243 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e0e30f-4b55-461b-8a44-fdc0d4ecf8f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:30.382880 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e0e30f-4b55-461b-8a44-fdc0d4ecf8f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:30.384468 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e0e30f-4b55-461b-8a44-fdc0d4ecf8f3 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:30.394366 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1307/2599] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:51:30 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:51:30.397926 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74caab5-87c1-4492-9b17-19c77bdcc869 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:30.407581 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74caab5-87c1-4492-9b17-19c77bdcc869 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:30.412623 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043f10ba-953a-4d23-b90c-e1e12431c709 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:30.413459 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043f10ba-953a-4d23-b90c-e1e12431c709 admin admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:30.413738 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043f10ba-953a-4d23-b90c-e1e12431c709 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:30.414012 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043f10ba-953a-4d23-b90c-e1e12431c709 admin admin] PATH_INFO: `/v3/roles` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:30.414637 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043f10ba-953a-4d23-b90c-e1e12431c709 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:30.416215 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043f10ba-953a-4d23-b90c-e1e12431c709 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:30.426646 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1293/2600] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:51:30 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:51:30.431413 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e0e30f-4b55-461b-8a44-fdc0d4ecf8f3 admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:30.441349 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e0e30f-4b55-461b-8a44-fdc0d4ecf8f3 admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:30.445930 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa2fc8d-0885-4785-b159-fcc60f8035e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:30.446579 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa2fc8d-0885-4785-b159-fcc60f8035e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:30.446858 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa2fc8d-0885-4785-b159-fcc60f8035e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:30.447128 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa2fc8d-0885-4785-b159-fcc60f8035e9 admin admin] PATH_INFO: `/v3/projects/d9aeb6ff763f49608bc509b9bdb8f4b0/users/743a92fd3bf14d21acff5e17f17a7309/roles/02476a5b82eb4b188c18a5d706a3ef68` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:30.461237 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa2fc8d-0885-4785-b159-fcc60f8035e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9aeb6ff763f49608bc509b9bdb8f4b0, user_id=743a92fd3bf14d21acff5e17f17a7309, role_id=02476a5b82eb4b188c18a5d706a3ef68)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:30.463931 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa2fc8d-0885-4785-b159-fcc60f8035e9 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:30.476890 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1308/2601] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 17:51:30 2022] PUT /identity/v3/projects/d9aeb6ff763f49608bc509b9bdb8f4b0/users/743a92fd3bf14d21acff5e17f17a7309/roles/02476a5b82eb4b188c18a5d706a3ef68 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:51:30.487282 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08308ad-6227-49a8-bc7e-1d60a34802d7 None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:30.487550 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08308ad-6227-49a8-bc7e-1d60a34802d7 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:30.487821 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08308ad-6227-49a8-bc7e-1d60a34802d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:30.507507 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:51:30.516987 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-f08308ad-6227-49a8-bc7e-1d60a34802d7 None None] MFA Rules not processed for user `743a92fd3bf14d21acff5e17f17a7309`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:51:30.518191 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-f08308ad-6227-49a8-bc7e-1d60a34802d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:30.584293 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1294/2602] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:51:30 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:51:30.588873 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa2fc8d-0885-4785-b159-fcc60f8035e9 admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:30.653721 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa2fc8d-0885-4785-b159-fcc60f8035e9 admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:30.658348 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885306f1-3a6b-4a6e-a2f7-d7fa356c957f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:30.658956 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885306f1-3a6b-4a6e-a2f7-d7fa356c957f admin admin] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:30.659232 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885306f1-3a6b-4a6e-a2f7-d7fa356c957f admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:30.659503 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885306f1-3a6b-4a6e-a2f7-d7fa356c957f admin admin] PATH_INFO: `/v3/projects` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:30.660145 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885306f1-3a6b-4a6e-a2f7-d7fa356c957f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:30.662705 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885306f1-3a6b-4a6e-a2f7-d7fa356c957f admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:30.689339 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1309/2603] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 17:51:30 2022] POST /identity/v3/projects => generated 369 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:51:30.693560 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08308ad-6227-49a8-bc7e-1d60a34802d7 None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:30.704333 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08308ad-6227-49a8-bc7e-1d60a34802d7 None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:30.765320 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6148bfc7-8bd8-458f-a192-179f53875b0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:30.765981 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6148bfc7-8bd8-458f-a192-179f53875b0c admin admin] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:30.766411 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6148bfc7-8bd8-458f-a192-179f53875b0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:30.766826 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6148bfc7-8bd8-458f-a192-179f53875b0c admin admin] PATH_INFO: `/v3/users` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:30.767626 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6148bfc7-8bd8-458f-a192-179f53875b0c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:30.773702 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6148bfc7-8bd8-458f-a192-179f53875b0c admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:30.776024 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885306f1-3a6b-4a6e-a2f7-d7fa356c957f admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:30.778049 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-885306f1-3a6b-4a6e-a2f7-d7fa356c957f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:30.803962 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1295/2604] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 17:51:30 2022] POST /identity/v3/users => generated 291 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:51:30.805075 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6148bfc7-8bd8-458f-a192-179f53875b0c admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:30.805701 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885306f1-3a6b-4a6e-a2f7-d7fa356c957f admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:30.810576 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e003d30-8a92-4b64-b9e8-55c82a59844f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:30.811211 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e003d30-8a92-4b64-b9e8-55c82a59844f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:30.811608 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e003d30-8a92-4b64-b9e8-55c82a59844f admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:30.811941 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e003d30-8a92-4b64-b9e8-55c82a59844f admin admin] PATH_INFO: `/v3/users/e9495aea97854ff3980d0b1e277b5617` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:30.813424 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e003d30-8a92-4b64-b9e8-55c82a59844f admin admin] RBAC: Authorizing `identity:delete_user(user_id=e9495aea97854ff3980d0b1e277b5617)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:30.815678 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e003d30-8a92-4b64-b9e8-55c82a59844f admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:30.817194 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6148bfc7-8bd8-458f-a192-179f53875b0c admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:30.824103 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de239391-8ac5-4c44-b042-d92dad4151cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:30.825162 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de239391-8ac5-4c44-b042-d92dad4151cd admin admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:30.825671 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de239391-8ac5-4c44-b042-d92dad4151cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:30.826235 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de239391-8ac5-4c44-b042-d92dad4151cd admin admin] PATH_INFO: `/v3/roles` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:30.827124 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de239391-8ac5-4c44-b042-d92dad4151cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:30.829820 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de239391-8ac5-4c44-b042-d92dad4151cd admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:30.852201 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1296/2605] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:51:30 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:51:30.854677 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de239391-8ac5-4c44-b042-d92dad4151cd admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:30.856315 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=85,keystone:DELETE=11,keystone:INSERT=4 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:51:30.870491 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de239391-8ac5-4c44-b042-d92dad4151cd admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:30.877595 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fbae42-b5a3-4ea8-bcda-13b88b50e667 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:30.878429 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fbae42-b5a3-4ea8-bcda-13b88b50e667 admin admin] REQUEST_METHOD: `PUT` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:30.878616 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fbae42-b5a3-4ea8-bcda-13b88b50e667 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:30.878983 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fbae42-b5a3-4ea8-bcda-13b88b50e667 admin admin] PATH_INFO: `/v3/projects/99a3f36f480844f28307757c71233146/users/e1a8857fde49440e9f94a0f6e3125163/roles/e411ad9eb96e4e8d85b59b5b700ada7b` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:30.881380 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-5e003d30-8a92-4b64-b9e8-55c82a59844f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e9495aea97854ff3980d0b1e277b5617'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:30.882114 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-5e003d30-8a92-4b64-b9e8-55c82a59844f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e9495aea97854ff3980d0b1e277b5617'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:30.898054 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fbae42-b5a3-4ea8-bcda-13b88b50e667 admin admin] RBAC: Authorizing `identity:create_grant(project_id=99a3f36f480844f28307757c71233146, user_id=e1a8857fde49440e9f94a0f6e3125163, role_id=e411ad9eb96e4e8d85b59b5b700ada7b)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:30.900663 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-5e003d30-8a92-4b64-b9e8-55c82a59844f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e9495aea97854ff3980d0b1e277b5617'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:30.900822 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fbae42-b5a3-4ea8-bcda-13b88b50e667 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:30.902851 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-5e003d30-8a92-4b64-b9e8-55c82a59844f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e9495aea97854ff3980d0b1e277b5617'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:30.920263 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-5e003d30-8a92-4b64-b9e8-55c82a59844f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e9495aea97854ff3980d0b1e277b5617'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:30.921708 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1297/2606] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 17:51:30 2022] PUT /identity/v3/projects/99a3f36f480844f28307757c71233146/users/e1a8857fde49440e9f94a0f6e3125163/roles/e411ad9eb96e4e8d85b59b5b700ada7b => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:51:30.926363 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fbae42-b5a3-4ea8-bcda-13b88b50e667 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:30.928448 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-f2fbae42-b5a3-4ea8-bcda-13b88b50e667 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:30.939060 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1310/2607] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 17:51:30 2022] DELETE /identity/v3/users/e9495aea97854ff3980d0b1e277b5617 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:51:30.943304 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0602958f-a187-48ce-9152-03541d75e5ea None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:30.943688 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0602958f-a187-48ce-9152-03541d75e5ea None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:30.944113 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0602958f-a187-48ce-9152-03541d75e5ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:30.966187 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-0602958f-a187-48ce-9152-03541d75e5ea None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:51:30.967826 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-0602958f-a187-48ce-9152-03541d75e5ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:31.001486 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:51:31.017969 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fbae42-b5a3-4ea8-bcda-13b88b50e667 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:31.042773 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1311/2608] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:51:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:51:31.043753 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0602958f-a187-48ce-9152-03541d75e5ea None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:31.045224 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-0602958f-a187-48ce-9152-03541d75e5ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:31.096725 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2040e9e8-d8e1-44c9-a568-0d25968db326 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:31.097437 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2040e9e8-d8e1-44c9-a568-0d25968db326 admin admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:31.097832 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2040e9e8-d8e1-44c9-a568-0d25968db326 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:31.098322 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2040e9e8-d8e1-44c9-a568-0d25968db326 admin admin] PATH_INFO: `/v3/roles` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:31.098888 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2040e9e8-d8e1-44c9-a568-0d25968db326 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:31.100642 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2040e9e8-d8e1-44c9-a568-0d25968db326 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:31.112284 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1298/2609] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:51:30 2022] GET /identity/v3/roles => generated 1420 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 17:51:31.116216 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0602958f-a187-48ce-9152-03541d75e5ea None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:31.116464 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2040e9e8-d8e1-44c9-a568-0d25968db326 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:31.120419 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7dbe3be-ef4c-4eac-9951-c3ee33357506 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43ad98a250a144099ccd18378aedd4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:31.121068 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dbe3be-ef4c-4eac-9951-c3ee33357506 service neutron] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:31.121530 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dbe3be-ef4c-4eac-9951-c3ee33357506 service neutron] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:31.121810 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dbe3be-ef4c-4eac-9951-c3ee33357506 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:31.126311 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2040e9e8-d8e1-44c9-a568-0d25968db326 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:31.132189 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebf9fef-271d-4f66-bd78-147b028dfef8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:31.132875 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebf9fef-271d-4f66-bd78-147b028dfef8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:31.133153 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebf9fef-271d-4f66-bd78-147b028dfef8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:31.133278 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7dbe3be-ef4c-4eac-9951-c3ee33357506 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:31.133868 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebf9fef-271d-4f66-bd78-147b028dfef8 admin admin] PATH_INFO: `/v3/projects/99a3f36f480844f28307757c71233146/users/e1a8857fde49440e9f94a0f6e3125163/roles/e411ad9eb96e4e8d85b59b5b700ada7b` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:31.134714 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7dbe3be-ef4c-4eac-9951-c3ee33357506 service neutron] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:31.135551 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebf9fef-271d-4f66-bd78-147b028dfef8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=99a3f36f480844f28307757c71233146, user_id=e1a8857fde49440e9f94a0f6e3125163, role_id=e411ad9eb96e4e8d85b59b5b700ada7b)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:31.138180 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebf9fef-271d-4f66-bd78-147b028dfef8 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:31.144504 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1312/2610] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:51:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:51:31.149850 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1299/2611] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 17:51:31 2022] PUT /identity/v3/projects/99a3f36f480844f28307757c71233146/users/e1a8857fde49440e9f94a0f6e3125163/roles/e411ad9eb96e4e8d85b59b5b700ada7b => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:51:31.160109 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3b0028-39ef-40b6-b1ed-a33db681286e None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:31.160409 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3b0028-39ef-40b6-b1ed-a33db681286e None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:31.160766 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3b0028-39ef-40b6-b1ed-a33db681286e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:31.191005 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-3d3b0028-39ef-40b6-b1ed-a33db681286e None None] MFA Rules not processed for user `e1a8857fde49440e9f94a0f6e3125163`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:51:31.192767 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-3d3b0028-39ef-40b6-b1ed-a33db681286e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:31.243736 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:51:31.284581 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1313/2612] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:51:31 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:51:31.292537 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebf9fef-271d-4f66-bd78-147b028dfef8 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:31.294452 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-2ebf9fef-271d-4f66-bd78-147b028dfef8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:31.367116 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebf9fef-271d-4f66-bd78-147b028dfef8 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:31.370986 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cf5d9b-b41d-497a-a3cc-4ab06cec3c15 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59be342541264942ad87f60ff4de6c8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:31.371697 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cf5d9b-b41d-497a-a3cc-4ab06cec3c15 service cinder] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:31.372037 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cf5d9b-b41d-497a-a3cc-4ab06cec3c15 service cinder] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:31.372393 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cf5d9b-b41d-497a-a3cc-4ab06cec3c15 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:31.382702 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cf5d9b-b41d-497a-a3cc-4ab06cec3c15 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:31.384188 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cf5d9b-b41d-497a-a3cc-4ab06cec3c15 service cinder] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:31.394146 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1300/2613] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:51:31 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:51:31.579064 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3b0028-39ef-40b6-b1ed-a33db681286e None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:31.580479 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-3d3b0028-39ef-40b6-b1ed-a33db681286e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:31.680342 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3b0028-39ef-40b6-b1ed-a33db681286e None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:31.685748 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e6a7ec-6d9b-48ae-906c-c4874818fd46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:31.686392 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e6a7ec-6d9b-48ae-906c-c4874818fd46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:31.686757 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e6a7ec-6d9b-48ae-906c-c4874818fd46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:31.687232 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e6a7ec-6d9b-48ae-906c-c4874818fd46 admin admin] PATH_INFO: `/v3/projects/4eded6b57db7437f82fbcb0a5a7f5c2a` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:31.688219 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e6a7ec-6d9b-48ae-906c-c4874818fd46 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4eded6b57db7437f82fbcb0a5a7f5c2a)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:31.690225 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e6a7ec-6d9b-48ae-906c-c4874818fd46 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:31.738224 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-64e6a7ec-6d9b-48ae-906c-c4874818fd46 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4eded6b57db7437f82fbcb0a5a7f5c2a', 'request_id': 'req-64e6a7ec-6d9b-48ae-906c-c4874818fd46'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:31.747691 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-64e6a7ec-6d9b-48ae-906c-c4874818fd46 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4eded6b57db7437f82fbcb0a5a7f5c2a', 'request_id': 'req-64e6a7ec-6d9b-48ae-906c-c4874818fd46'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:31.755070 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-64e6a7ec-6d9b-48ae-906c-c4874818fd46 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4eded6b57db7437f82fbcb0a5a7f5c2a', 'request_id': 'req-64e6a7ec-6d9b-48ae-906c-c4874818fd46'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:31.765963 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-64e6a7ec-6d9b-48ae-906c-c4874818fd46 admin admin] The token cache is being invalidate because project 4eded6b57db7437f82fbcb0a5a7f5c2a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81758) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:51:31.766181 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-64e6a7ec-6d9b-48ae-906c-c4874818fd46 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:31.766859 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-64e6a7ec-6d9b-48ae-906c-c4874818fd46 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:31.768704 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1314/2614] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 17:51:31 2022] DELETE /identity/v3/projects/4eded6b57db7437f82fbcb0a5a7f5c2a => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:51:31.772728 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cf5d9b-b41d-497a-a3cc-4ab06cec3c15 service cinder] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:31.783520 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cf5d9b-b41d-497a-a3cc-4ab06cec3c15 service cinder] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:31.843015 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0d80b2-4084-4136-91ca-f8e61d891cb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:31.843677 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0d80b2-4084-4136-91ca-f8e61d891cb9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:31.843959 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0d80b2-4084-4136-91ca-f8e61d891cb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:31.844228 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0d80b2-4084-4136-91ca-f8e61d891cb9 admin admin] PATH_INFO: `/v3/users/28f76d528864447f90425edf37ac6121` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:31.845387 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0d80b2-4084-4136-91ca-f8e61d891cb9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=28f76d528864447f90425edf37ac6121)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:31.847458 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0d80b2-4084-4136-91ca-f8e61d891cb9 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:31.883893 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:51:31.898753 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-8b0d80b2-4084-4136-91ca-f8e61d891cb9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '28f76d528864447f90425edf37ac6121'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:31.899399 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-8b0d80b2-4084-4136-91ca-f8e61d891cb9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '28f76d528864447f90425edf37ac6121'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:31.906564 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-8b0d80b2-4084-4136-91ca-f8e61d891cb9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '28f76d528864447f90425edf37ac6121'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:31.907698 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-8b0d80b2-4084-4136-91ca-f8e61d891cb9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '28f76d528864447f90425edf37ac6121'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:31.915692 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-8b0d80b2-4084-4136-91ca-f8e61d891cb9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '28f76d528864447f90425edf37ac6121'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:31.941461 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1301/2615] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 17:51:31 2022] DELETE /identity/v3/users/28f76d528864447f90425edf37ac6121 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:51:32.267061 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e6a7ec-6d9b-48ae-906c-c4874818fd46 admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:32.270563 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-64e6a7ec-6d9b-48ae-906c-c4874818fd46 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:32.296162 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e6a7ec-6d9b-48ae-906c-c4874818fd46 admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:32.359454 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65650bcf-c916-48bd-ab03-8796d2213803 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:32.361170 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65650bcf-c916-48bd-ab03-8796d2213803 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:32.361456 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65650bcf-c916-48bd-ab03-8796d2213803 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:32.361733 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65650bcf-c916-48bd-ab03-8796d2213803 admin admin] PATH_INFO: `/v3/projects/f0d418521bde41f39217829e4b8ed11c` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:32.362863 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65650bcf-c916-48bd-ab03-8796d2213803 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f0d418521bde41f39217829e4b8ed11c)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:32.364894 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65650bcf-c916-48bd-ab03-8796d2213803 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:32.372615 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:51:32.418508 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-65650bcf-c916-48bd-ab03-8796d2213803 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f0d418521bde41f39217829e4b8ed11c', 'request_id': 'req-65650bcf-c916-48bd-ab03-8796d2213803'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:32.430995 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-65650bcf-c916-48bd-ab03-8796d2213803 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f0d418521bde41f39217829e4b8ed11c', 'request_id': 'req-65650bcf-c916-48bd-ab03-8796d2213803'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:32.443822 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-65650bcf-c916-48bd-ab03-8796d2213803 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f0d418521bde41f39217829e4b8ed11c', 'request_id': 'req-65650bcf-c916-48bd-ab03-8796d2213803'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:32.460438 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-65650bcf-c916-48bd-ab03-8796d2213803 admin admin] The token cache is being invalidate because project f0d418521bde41f39217829e4b8ed11c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81758) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:51:32.460997 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-65650bcf-c916-48bd-ab03-8796d2213803 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:32.461668 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-65650bcf-c916-48bd-ab03-8796d2213803 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:32.463215 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1315/2616] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 17:51:32 2022] DELETE /identity/v3/projects/f0d418521bde41f39217829e4b8ed11c => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:51:32.469010 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f091333c-ce40-4281-baab-7cf441a79c19 None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:32.469485 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f091333c-ce40-4281-baab-7cf441a79c19 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:32.469725 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f091333c-ce40-4281-baab-7cf441a79c19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:32.490393 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-f091333c-ce40-4281-baab-7cf441a79c19 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:51:32.491525 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-f091333c-ce40-4281-baab-7cf441a79c19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:32.552490 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1302/2617] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:51:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:51:32.562048 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86c619f-a6ac-4c6e-b2ff-4ad9a46d858d None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:32.562385 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86c619f-a6ac-4c6e-b2ff-4ad9a46d858d None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:32.562666 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86c619f-a6ac-4c6e-b2ff-4ad9a46d858d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:32.583770 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-b86c619f-a6ac-4c6e-b2ff-4ad9a46d858d None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:51:32.585029 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-b86c619f-a6ac-4c6e-b2ff-4ad9a46d858d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:32.593129 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1316/2618] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:51:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:51:32.597319 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f091333c-ce40-4281-baab-7cf441a79c19 None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:32.609674 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f091333c-ce40-4281-baab-7cf441a79c19 None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:32.614239 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52896e1a-f0b4-4bb4-91f4-25c4683bb170 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:32.614840 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52896e1a-f0b4-4bb4-91f4-25c4683bb170 admin admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:32.615123 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52896e1a-f0b4-4bb4-91f4-25c4683bb170 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:32.615392 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52896e1a-f0b4-4bb4-91f4-25c4683bb170 admin admin] PATH_INFO: `/v3/domains` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:32.616051 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52896e1a-f0b4-4bb4-91f4-25c4683bb170 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:51:32.616337 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52896e1a-f0b4-4bb4-91f4-25c4683bb170 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:32.617952 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52896e1a-f0b4-4bb4-91f4-25c4683bb170 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:32.629863 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1303/2619] 10.222.0.45 () {58 vars in 1153 bytes} [Wed Aug 24 17:51:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:51:32.634279 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86c619f-a6ac-4c6e-b2ff-4ad9a46d858d None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:32.644131 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86c619f-a6ac-4c6e-b2ff-4ad9a46d858d None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:32.649784 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b09b55-578d-4e4f-8471-0de06c13eb87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:32.650413 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b09b55-578d-4e4f-8471-0de06c13eb87 admin admin] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:32.650719 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b09b55-578d-4e4f-8471-0de06c13eb87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:32.650962 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b09b55-578d-4e4f-8471-0de06c13eb87 admin admin] PATH_INFO: `/v3/projects` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:32.651587 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b09b55-578d-4e4f-8471-0de06c13eb87 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:32.656327 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b09b55-578d-4e4f-8471-0de06c13eb87 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:32.692955 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1317/2620] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 17:51:32 2022] POST /identity/v3/projects => generated 361 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:51:32.756870 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52896e1a-f0b4-4bb4-91f4-25c4683bb170 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:32.767315 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52896e1a-f0b4-4bb4-91f4-25c4683bb170 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:32.834466 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd18de61-1908-416b-b0b9-14eab3828857 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:32.835063 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd18de61-1908-416b-b0b9-14eab3828857 admin admin] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:32.835341 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd18de61-1908-416b-b0b9-14eab3828857 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:32.835710 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd18de61-1908-416b-b0b9-14eab3828857 admin admin] PATH_INFO: `/v3/users` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:32.836220 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd18de61-1908-416b-b0b9-14eab3828857 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:32.838161 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd18de61-1908-416b-b0b9-14eab3828857 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:32.859306 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1304/2621] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 17:51:32 2022] POST /identity/v3/users => generated 281 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:51:32.863527 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b09b55-578d-4e4f-8471-0de06c13eb87 admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:32.873697 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b09b55-578d-4e4f-8471-0de06c13eb87 admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:32.878280 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243009d1-2515-4ac8-887a-0f824e436a31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:32.878881 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243009d1-2515-4ac8-887a-0f824e436a31 admin admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:32.879689 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243009d1-2515-4ac8-887a-0f824e436a31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:32.879963 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243009d1-2515-4ac8-887a-0f824e436a31 admin admin] PATH_INFO: `/v3/roles` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:32.880627 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243009d1-2515-4ac8-887a-0f824e436a31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:32.882196 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243009d1-2515-4ac8-887a-0f824e436a31 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:32.886570 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd18de61-1908-416b-b0b9-14eab3828857 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:32.888838 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-bd18de61-1908-416b-b0b9-14eab3828857 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:32.895260 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1318/2622] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:51:32 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:51:32.896188 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243009d1-2515-4ac8-887a-0f824e436a31 admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:32.901288 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd18de61-1908-416b-b0b9-14eab3828857 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:32.905616 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243009d1-2515-4ac8-887a-0f824e436a31 admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:32.906406 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d939cc-73b4-4b08-afec-e1e8187e1c48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:32.907036 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d939cc-73b4-4b08-afec-e1e8187e1c48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:32.907346 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d939cc-73b4-4b08-afec-e1e8187e1c48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:32.907623 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d939cc-73b4-4b08-afec-e1e8187e1c48 admin admin] PATH_INFO: `/v3/users/743a92fd3bf14d21acff5e17f17a7309` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:32.908810 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d939cc-73b4-4b08-afec-e1e8187e1c48 admin admin] RBAC: Authorizing `identity:delete_user(user_id=743a92fd3bf14d21acff5e17f17a7309)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:32.910529 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2b5947-29db-49e2-a766-7a0d2809cbcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:32.910742 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d939cc-73b4-4b08-afec-e1e8187e1c48 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:32.911093 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2b5947-29db-49e2-a766-7a0d2809cbcd admin admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:32.911367 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2b5947-29db-49e2-a766-7a0d2809cbcd admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:32.911633 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2b5947-29db-49e2-a766-7a0d2809cbcd admin admin] PATH_INFO: `/v3/roles` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:32.912254 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2b5947-29db-49e2-a766-7a0d2809cbcd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:32.914108 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2b5947-29db-49e2-a766-7a0d2809cbcd admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:32.923812 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1319/2623] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:51:32 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:51:32.928539 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2b5947-29db-49e2-a766-7a0d2809cbcd admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:32.929188 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:51:32.938430 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2b5947-29db-49e2-a766-7a0d2809cbcd admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:32.943986 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84614ec-da17-4f0e-a9fd-59dbc3dc4061 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:32.944652 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84614ec-da17-4f0e-a9fd-59dbc3dc4061 admin admin] REQUEST_METHOD: `PUT` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:32.944961 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84614ec-da17-4f0e-a9fd-59dbc3dc4061 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:32.945206 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84614ec-da17-4f0e-a9fd-59dbc3dc4061 admin admin] PATH_INFO: `/v3/projects/15745a293fb64df9b4cd87d942ccb42e/users/a3e3aec401244422a952de05b41b51d0/roles/02476a5b82eb4b188c18a5d706a3ef68` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:32.960060 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84614ec-da17-4f0e-a9fd-59dbc3dc4061 admin admin] RBAC: Authorizing `identity:create_grant(project_id=15745a293fb64df9b4cd87d942ccb42e, user_id=a3e3aec401244422a952de05b41b51d0, role_id=02476a5b82eb4b188c18a5d706a3ef68)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:32.961914 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-58d939cc-73b4-4b08-afec-e1e8187e1c48 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '743a92fd3bf14d21acff5e17f17a7309'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:32.962592 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-58d939cc-73b4-4b08-afec-e1e8187e1c48 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '743a92fd3bf14d21acff5e17f17a7309'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:32.962895 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84614ec-da17-4f0e-a9fd-59dbc3dc4061 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:32.969678 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-58d939cc-73b4-4b08-afec-e1e8187e1c48 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '743a92fd3bf14d21acff5e17f17a7309'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:32.970397 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-58d939cc-73b4-4b08-afec-e1e8187e1c48 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '743a92fd3bf14d21acff5e17f17a7309'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:32.975508 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1320/2624] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 17:51:32 2022] PUT /identity/v3/projects/15745a293fb64df9b4cd87d942ccb42e/users/a3e3aec401244422a952de05b41b51d0/roles/02476a5b82eb4b188c18a5d706a3ef68 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:51:32.983660 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-58d939cc-73b4-4b08-afec-e1e8187e1c48 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '743a92fd3bf14d21acff5e17f17a7309'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:32.986939 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a759602b-8c11-40f9-a845-f39b4c8b3326 None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:32.987227 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a759602b-8c11-40f9-a845-f39b4c8b3326 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:32.987558 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a759602b-8c11-40f9-a845-f39b4c8b3326 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:33.002734 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1305/2625] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 17:51:32 2022] DELETE /identity/v3/users/743a92fd3bf14d21acff5e17f17a7309 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:51:33.007750 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa725a6-0c1c-49f7-9258-788beee05b16 None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:33.008040 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa725a6-0c1c-49f7-9258-788beee05b16 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:33.008317 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa725a6-0c1c-49f7-9258-788beee05b16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:33.017195 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-a759602b-8c11-40f9-a845-f39b4c8b3326 None None] MFA Rules not processed for user `a3e3aec401244422a952de05b41b51d0`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:51:33.018418 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-a759602b-8c11-40f9-a845-f39b4c8b3326 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:33.026716 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-9aa725a6-0c1c-49f7-9258-788beee05b16 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:51:33.027874 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-9aa725a6-0c1c-49f7-9258-788beee05b16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:33.082893 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1321/2626] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:51:32 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:51:33.085687 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1306/2627] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:51:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:51:33.095537 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a759602b-8c11-40f9-a845-f39b4c8b3326 None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:33.096946 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-a759602b-8c11-40f9-a845-f39b4c8b3326 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:33.108445 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:51:33.117137 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa725a6-0c1c-49f7-9258-788beee05b16 None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:33.118615 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-9aa725a6-0c1c-49f7-9258-788beee05b16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:33.157624 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a759602b-8c11-40f9-a845-f39b4c8b3326 None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:33.161520 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2405c5-ccb5-44ca-bd1b-fa170f8f861f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43ad98a250a144099ccd18378aedd4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:33.162125 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2405c5-ccb5-44ca-bd1b-fa170f8f861f service neutron] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:33.162406 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2405c5-ccb5-44ca-bd1b-fa170f8f861f service neutron] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:33.162941 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2405c5-ccb5-44ca-bd1b-fa170f8f861f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:33.172474 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2405c5-ccb5-44ca-bd1b-fa170f8f861f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:33.173935 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2405c5-ccb5-44ca-bd1b-fa170f8f861f service neutron] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:33.175020 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa725a6-0c1c-49f7-9258-788beee05b16 None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:33.179333 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3928e3df-b405-4641-b8e6-12ca62c4f07a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59be342541264942ad87f60ff4de6c8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:33.179943 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3928e3df-b405-4641-b8e6-12ca62c4f07a service cinder] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:33.180217 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3928e3df-b405-4641-b8e6-12ca62c4f07a service cinder] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:33.180492 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3928e3df-b405-4641-b8e6-12ca62c4f07a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:33.183861 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1322/2628] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:51:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:51:33.189251 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3928e3df-b405-4641-b8e6-12ca62c4f07a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:33.190682 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3928e3df-b405-4641-b8e6-12ca62c4f07a service cinder] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:33.200239 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1307/2629] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:51:33 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 17:51:33.215548 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2405c5-ccb5-44ca-bd1b-fa170f8f861f service neutron] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:33.217058 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-af2405c5-ccb5-44ca-bd1b-fa170f8f861f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:33.230111 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2405c5-ccb5-44ca-bd1b-fa170f8f861f service neutron] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:33.234852 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08db5940-873c-4e08-b765-afc24bff1a1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:33.235419 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08db5940-873c-4e08-b765-afc24bff1a1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:33.235701 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08db5940-873c-4e08-b765-afc24bff1a1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:33.235979 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08db5940-873c-4e08-b765-afc24bff1a1a admin admin] PATH_INFO: `/v3/users/a3e3aec401244422a952de05b41b51d0` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:33.237160 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08db5940-873c-4e08-b765-afc24bff1a1a admin admin] RBAC: Authorizing `identity:delete_user(user_id=a3e3aec401244422a952de05b41b51d0)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:33.239616 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08db5940-873c-4e08-b765-afc24bff1a1a admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:33.293164 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-08db5940-873c-4e08-b765-afc24bff1a1a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a3e3aec401244422a952de05b41b51d0'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:33.293951 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-08db5940-873c-4e08-b765-afc24bff1a1a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a3e3aec401244422a952de05b41b51d0'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:33.304905 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-08db5940-873c-4e08-b765-afc24bff1a1a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a3e3aec401244422a952de05b41b51d0'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:33.305646 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-08db5940-873c-4e08-b765-afc24bff1a1a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a3e3aec401244422a952de05b41b51d0'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:33.318989 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-08db5940-873c-4e08-b765-afc24bff1a1a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a3e3aec401244422a952de05b41b51d0'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:33.333107 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1323/2630] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 17:51:33 2022] DELETE /identity/v3/users/a3e3aec401244422a952de05b41b51d0 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:51:33.337814 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635df460-5d2a-43b4-82d8-ea7f405a9145 None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:33.338723 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635df460-5d2a-43b4-82d8-ea7f405a9145 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:33.339376 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635df460-5d2a-43b4-82d8-ea7f405a9145 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:33.363258 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-635df460-5d2a-43b4-82d8-ea7f405a9145 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:51:33.365637 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-635df460-5d2a-43b4-82d8-ea7f405a9145 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:33.405844 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:51:33.440796 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1308/2631] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:51:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:51:33.453931 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08db5940-873c-4e08-b765-afc24bff1a1a admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:33.455348 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-08db5940-873c-4e08-b765-afc24bff1a1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:33.485556 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635df460-5d2a-43b4-82d8-ea7f405a9145 None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:33.487822 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-635df460-5d2a-43b4-82d8-ea7f405a9145 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:33.500976 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635df460-5d2a-43b4-82d8-ea7f405a9145 None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:33.505718 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ead4dd9-66e2-4c37-8c78-86ac3545aa14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:33.506357 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ead4dd9-66e2-4c37-8c78-86ac3545aa14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:33.506637 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ead4dd9-66e2-4c37-8c78-86ac3545aa14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:33.506909 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ead4dd9-66e2-4c37-8c78-86ac3545aa14 admin admin] PATH_INFO: `/v3/projects/d9aeb6ff763f49608bc509b9bdb8f4b0` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:33.508043 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ead4dd9-66e2-4c37-8c78-86ac3545aa14 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d9aeb6ff763f49608bc509b9bdb8f4b0)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:33.510003 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ead4dd9-66e2-4c37-8c78-86ac3545aa14 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:33.522975 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08db5940-873c-4e08-b765-afc24bff1a1a admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:33.526943 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c82be9-8048-46e3-87ae-23f6c438c69a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43ad98a250a144099ccd18378aedd4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:33.527546 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c82be9-8048-46e3-87ae-23f6c438c69a service neutron] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:33.527820 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c82be9-8048-46e3-87ae-23f6c438c69a service neutron] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:33.528096 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c82be9-8048-46e3-87ae-23f6c438c69a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:33.538979 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c82be9-8048-46e3-87ae-23f6c438c69a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:33.539874 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c82be9-8048-46e3-87ae-23f6c438c69a service neutron] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:33.549745 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1324/2632] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:51:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:51:33.560239 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-0ead4dd9-66e2-4c37-8c78-86ac3545aa14 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd9aeb6ff763f49608bc509b9bdb8f4b0', 'request_id': 'req-0ead4dd9-66e2-4c37-8c78-86ac3545aa14'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:33.566188 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-0ead4dd9-66e2-4c37-8c78-86ac3545aa14 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd9aeb6ff763f49608bc509b9bdb8f4b0', 'request_id': 'req-0ead4dd9-66e2-4c37-8c78-86ac3545aa14'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:33.572722 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-0ead4dd9-66e2-4c37-8c78-86ac3545aa14 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd9aeb6ff763f49608bc509b9bdb8f4b0', 'request_id': 'req-0ead4dd9-66e2-4c37-8c78-86ac3545aa14'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:33.586751 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-0ead4dd9-66e2-4c37-8c78-86ac3545aa14 admin admin] The token cache is being invalidate because project d9aeb6ff763f49608bc509b9bdb8f4b0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81759) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:51:33.587129 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-0ead4dd9-66e2-4c37-8c78-86ac3545aa14 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:33.587890 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-0ead4dd9-66e2-4c37-8c78-86ac3545aa14 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:33.589518 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1309/2633] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 17:51:33 2022] DELETE /identity/v3/projects/d9aeb6ff763f49608bc509b9bdb8f4b0 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:51:33.593444 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c82be9-8048-46e3-87ae-23f6c438c69a service neutron] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:33.604044 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c82be9-8048-46e3-87ae-23f6c438c69a service neutron] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:33.628198 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:51:33.665275 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a35ab70-2e64-4412-ac91-79875680beda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:33.666274 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a35ab70-2e64-4412-ac91-79875680beda admin admin] REQUEST_METHOD: `DELETE` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:33.666274 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a35ab70-2e64-4412-ac91-79875680beda admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:33.666487 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a35ab70-2e64-4412-ac91-79875680beda admin admin] PATH_INFO: `/v3/users/e1a8857fde49440e9f94a0f6e3125163` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:33.667616 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a35ab70-2e64-4412-ac91-79875680beda admin admin] RBAC: Authorizing `identity:delete_user(user_id=e1a8857fde49440e9f94a0f6e3125163)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:33.669620 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a35ab70-2e64-4412-ac91-79875680beda admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:33.720796 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-1a35ab70-2e64-4412-ac91-79875680beda admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e1a8857fde49440e9f94a0f6e3125163'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:33.721704 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-1a35ab70-2e64-4412-ac91-79875680beda admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e1a8857fde49440e9f94a0f6e3125163'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:33.730622 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-1a35ab70-2e64-4412-ac91-79875680beda admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e1a8857fde49440e9f94a0f6e3125163'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:33.731347 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-1a35ab70-2e64-4412-ac91-79875680beda admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e1a8857fde49440e9f94a0f6e3125163'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:33.744084 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-1a35ab70-2e64-4412-ac91-79875680beda admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e1a8857fde49440e9f94a0f6e3125163'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:33.757994 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1325/2634] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 17:51:33 2022] DELETE /identity/v3/users/e1a8857fde49440e9f94a0f6e3125163 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:51:33.835050 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ead4dd9-66e2-4c37-8c78-86ac3545aa14 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:33.837112 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-0ead4dd9-66e2-4c37-8c78-86ac3545aa14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:33.853321 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ead4dd9-66e2-4c37-8c78-86ac3545aa14 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:33.934082 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed456c37-86b4-4e56-ba22-3a24fb0a3323 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:33.934907 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed456c37-86b4-4e56-ba22-3a24fb0a3323 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:33.935288 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed456c37-86b4-4e56-ba22-3a24fb0a3323 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:33.935666 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed456c37-86b4-4e56-ba22-3a24fb0a3323 admin admin] PATH_INFO: `/v3/projects/15745a293fb64df9b4cd87d942ccb42e` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:33.937164 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed456c37-86b4-4e56-ba22-3a24fb0a3323 admin admin] RBAC: Authorizing `identity:delete_project(project_id=15745a293fb64df9b4cd87d942ccb42e)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:33.939228 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed456c37-86b4-4e56-ba22-3a24fb0a3323 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:33.995839 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-ed456c37-86b4-4e56-ba22-3a24fb0a3323 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '15745a293fb64df9b4cd87d942ccb42e', 'request_id': 'req-ed456c37-86b4-4e56-ba22-3a24fb0a3323'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:34.002160 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-ed456c37-86b4-4e56-ba22-3a24fb0a3323 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '15745a293fb64df9b4cd87d942ccb42e', 'request_id': 'req-ed456c37-86b4-4e56-ba22-3a24fb0a3323'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:34.008541 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-ed456c37-86b4-4e56-ba22-3a24fb0a3323 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '15745a293fb64df9b4cd87d942ccb42e', 'request_id': 'req-ed456c37-86b4-4e56-ba22-3a24fb0a3323'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:34.010412 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:51:34.023885 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-ed456c37-86b4-4e56-ba22-3a24fb0a3323 admin admin] The token cache is being invalidate because project 15745a293fb64df9b4cd87d942ccb42e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81759) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:51:34.024747 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-ed456c37-86b4-4e56-ba22-3a24fb0a3323 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:34.025566 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-ed456c37-86b4-4e56-ba22-3a24fb0a3323 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:34.028586 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1310/2635] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 17:51:33 2022] DELETE /identity/v3/projects/15745a293fb64df9b4cd87d942ccb42e => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:51:34.033393 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7170bc-be7f-4cc4-afd9-b55892242a64 None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:34.033683 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7170bc-be7f-4cc4-afd9-b55892242a64 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:34.034007 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7170bc-be7f-4cc4-afd9-b55892242a64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:34.047214 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed456c37-86b4-4e56-ba22-3a24fb0a3323 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:34.049209 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-ed456c37-86b4-4e56-ba22-3a24fb0a3323 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:34.056442 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-8e7170bc-be7f-4cc4-afd9-b55892242a64 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:51:34.058712 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-8e7170bc-be7f-4cc4-afd9-b55892242a64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:34.119322 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed456c37-86b4-4e56-ba22-3a24fb0a3323 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:34.123990 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8902a4be-70fc-4dc5-aa5a-8d4a69c55365 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:34.124847 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8902a4be-70fc-4dc5-aa5a-8d4a69c55365 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:34.125126 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8902a4be-70fc-4dc5-aa5a-8d4a69c55365 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:34.125398 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8902a4be-70fc-4dc5-aa5a-8d4a69c55365 admin admin] PATH_INFO: `/v3/projects/99a3f36f480844f28307757c71233146` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:34.126407 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1326/2636] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:51:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:51:34.126579 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8902a4be-70fc-4dc5-aa5a-8d4a69c55365 admin admin] RBAC: Authorizing `identity:delete_project(project_id=99a3f36f480844f28307757c71233146)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:34.128407 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8902a4be-70fc-4dc5-aa5a-8d4a69c55365 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:34.134630 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d31ad06-6d80-41d7-ad26-28f142b3e4c7 None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:34.134945 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d31ad06-6d80-41d7-ad26-28f142b3e4c7 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:34.135231 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d31ad06-6d80-41d7-ad26-28f142b3e4c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:34.156062 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-1d31ad06-6d80-41d7-ad26-28f142b3e4c7 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:51:34.157298 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-1d31ad06-6d80-41d7-ad26-28f142b3e4c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:34.165942 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1327/2637] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:51:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:51:34.170034 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d31ad06-6d80-41d7-ad26-28f142b3e4c7 None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:34.176371 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-8902a4be-70fc-4dc5-aa5a-8d4a69c55365 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '99a3f36f480844f28307757c71233146', 'request_id': 'req-8902a4be-70fc-4dc5-aa5a-8d4a69c55365'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:34.178915 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d31ad06-6d80-41d7-ad26-28f142b3e4c7 None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:34.187151 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-8902a4be-70fc-4dc5-aa5a-8d4a69c55365 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '99a3f36f480844f28307757c71233146', 'request_id': 'req-8902a4be-70fc-4dc5-aa5a-8d4a69c55365'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:34.193985 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-8902a4be-70fc-4dc5-aa5a-8d4a69c55365 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '99a3f36f480844f28307757c71233146', 'request_id': 'req-8902a4be-70fc-4dc5-aa5a-8d4a69c55365'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:34.207094 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-8902a4be-70fc-4dc5-aa5a-8d4a69c55365 admin admin] The token cache is being invalidate because project 99a3f36f480844f28307757c71233146 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81759) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:51:34.207895 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-8902a4be-70fc-4dc5-aa5a-8d4a69c55365 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:34.208580 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-8902a4be-70fc-4dc5-aa5a-8d4a69c55365 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:34.210106 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1311/2638] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 17:51:34 2022] DELETE /identity/v3/projects/99a3f36f480844f28307757c71233146 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:51:34.214888 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee30092-bdd1-43ae-92b1-a8ebac3e70dc None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:34.215176 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee30092-bdd1-43ae-92b1-a8ebac3e70dc None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:34.215452 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee30092-bdd1-43ae-92b1-a8ebac3e70dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:34.220426 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:51:34.235756 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-aee30092-bdd1-43ae-92b1-a8ebac3e70dc None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:51:34.236988 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-aee30092-bdd1-43ae-92b1-a8ebac3e70dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:34.240673 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9863654-969e-4de0-bf0b-1be808e0ac04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:34.241287 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9863654-969e-4de0-bf0b-1be808e0ac04 admin admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:34.241570 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9863654-969e-4de0-bf0b-1be808e0ac04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:34.241844 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9863654-969e-4de0-bf0b-1be808e0ac04 admin admin] PATH_INFO: `/v3/domains` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:34.242488 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9863654-969e-4de0-bf0b-1be808e0ac04 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:51:34.242793 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9863654-969e-4de0-bf0b-1be808e0ac04 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:34.244396 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9863654-969e-4de0-bf0b-1be808e0ac04 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:34.244923 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1312/2639] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:51:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:51:34.252632 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a718d65-424f-4ac2-a19e-333cd86ca99e None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:34.252845 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a718d65-424f-4ac2-a19e-333cd86ca99e None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:34.253179 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a718d65-424f-4ac2-a19e-333cd86ca99e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:34.260779 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1328/2640] 10.222.0.45 () {58 vars in 1153 bytes} [Wed Aug 24 17:51:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:51:34.265268 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9863654-969e-4de0-bf0b-1be808e0ac04 admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:34.266945 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-b9863654-969e-4de0-bf0b-1be808e0ac04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:34.274191 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-8a718d65-424f-4ac2-a19e-333cd86ca99e None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:51:34.275344 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-8a718d65-424f-4ac2-a19e-333cd86ca99e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:34.279542 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9863654-969e-4de0-bf0b-1be808e0ac04 admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:34.283360 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1313/2641] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:51:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:51:34.284151 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d80076b-6e2d-45ae-9355-971e8bef56ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:34.284780 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d80076b-6e2d-45ae-9355-971e8bef56ef admin admin] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:34.285114 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d80076b-6e2d-45ae-9355-971e8bef56ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:34.285397 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d80076b-6e2d-45ae-9355-971e8bef56ef admin admin] PATH_INFO: `/v3/projects` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:34.286006 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d80076b-6e2d-45ae-9355-971e8bef56ef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:34.287825 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a718d65-424f-4ac2-a19e-333cd86ca99e None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:34.288183 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d80076b-6e2d-45ae-9355-971e8bef56ef admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:34.296901 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a718d65-424f-4ac2-a19e-333cd86ca99e None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:34.302182 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8dcacc4-b731-4c35-b769-8ac4f27394d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:34.302997 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dcacc4-b731-4c35-b769-8ac4f27394d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:34.303394 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dcacc4-b731-4c35-b769-8ac4f27394d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:34.303736 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dcacc4-b731-4c35-b769-8ac4f27394d7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:34.304619 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8dcacc4-b731-4c35-b769-8ac4f27394d7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:51:34.305013 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8dcacc4-b731-4c35-b769-8ac4f27394d7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:34.306653 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8dcacc4-b731-4c35-b769-8ac4f27394d7 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:34.316252 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1329/2642] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 17:51:34 2022] POST /identity/v3/projects => generated 399 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:51:34.319343 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1314/2643] 10.222.0.45 () {58 vars in 1153 bytes} [Wed Aug 24 17:51:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:51:34.321011 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d80076b-6e2d-45ae-9355-971e8bef56ef admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:34.323428 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8dcacc4-b731-4c35-b769-8ac4f27394d7 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:34.333746 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8dcacc4-b731-4c35-b769-8ac4f27394d7 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:34.382091 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:INSERT=2,keystone:DELETE=8,keystone:SELECT=90 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:51:34.384276 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d80076b-6e2d-45ae-9355-971e8bef56ef admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:34.389464 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a86ee97-8ae6-443a-be45-53cdc2ae7fc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:34.389881 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4a8085-49b4-4a76-8658-07df466a52ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:34.390074 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a86ee97-8ae6-443a-be45-53cdc2ae7fc1 admin admin] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:34.390355 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a86ee97-8ae6-443a-be45-53cdc2ae7fc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:34.390681 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a86ee97-8ae6-443a-be45-53cdc2ae7fc1 admin admin] PATH_INFO: `/v3/users` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:34.391001 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4a8085-49b4-4a76-8658-07df466a52ef admin admin] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:34.391245 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a86ee97-8ae6-443a-be45-53cdc2ae7fc1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:34.391361 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4a8085-49b4-4a76-8658-07df466a52ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:34.391632 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4a8085-49b4-4a76-8658-07df466a52ef admin admin] PATH_INFO: `/v3/projects` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:34.392318 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4a8085-49b4-4a76-8658-07df466a52ef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:34.393231 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a86ee97-8ae6-443a-be45-53cdc2ae7fc1 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:34.395254 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4a8085-49b4-4a76-8658-07df466a52ef admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:34.417525 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1330/2644] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 17:51:34 2022] POST /identity/v3/users => generated 300 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:51:34.418643 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a86ee97-8ae6-443a-be45-53cdc2ae7fc1 admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:34.427098 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1315/2645] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 17:51:34 2022] POST /identity/v3/projects => generated 377 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:51:34.431083 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4a8085-49b4-4a76-8658-07df466a52ef admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:34.432281 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a86ee97-8ae6-443a-be45-53cdc2ae7fc1 admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:34.487820 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc21c8d0-969a-4819-9cf2-4d7baf4af308 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:34.489141 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc21c8d0-969a-4819-9cf2-4d7baf4af308 admin admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:34.489289 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc21c8d0-969a-4819-9cf2-4d7baf4af308 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:34.489496 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc21c8d0-969a-4819-9cf2-4d7baf4af308 admin admin] PATH_INFO: `/v3/roles` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:34.490133 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc21c8d0-969a-4819-9cf2-4d7baf4af308 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:34.491719 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc21c8d0-969a-4819-9cf2-4d7baf4af308 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:34.493389 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4a8085-49b4-4a76-8658-07df466a52ef admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:34.498138 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e44d42e-09ec-4983-af83-7c48dbce7d12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:34.505980 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e44d42e-09ec-4983-af83-7c48dbce7d12 admin admin] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:34.505980 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e44d42e-09ec-4983-af83-7c48dbce7d12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:34.505980 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e44d42e-09ec-4983-af83-7c48dbce7d12 admin admin] PATH_INFO: `/v3/users` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:34.505980 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e44d42e-09ec-4983-af83-7c48dbce7d12 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:34.506418 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1331/2646] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:51:34 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 17:51:34.506516 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e44d42e-09ec-4983-af83-7c48dbce7d12 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:34.507217 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc21c8d0-969a-4819-9cf2-4d7baf4af308 admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:34.516859 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc21c8d0-969a-4819-9cf2-4d7baf4af308 admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:34.521910 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b4deee-5784-4d2f-bdab-8fab2eb39eb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:34.522494 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b4deee-5784-4d2f-bdab-8fab2eb39eb6 admin admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:34.522798 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b4deee-5784-4d2f-bdab-8fab2eb39eb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:34.523069 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b4deee-5784-4d2f-bdab-8fab2eb39eb6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:34.523723 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b4deee-5784-4d2f-bdab-8fab2eb39eb6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:34.528272 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1316/2647] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 17:51:34 2022] POST /identity/v3/users => generated 289 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:51:34.528506 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b4deee-5784-4d2f-bdab-8fab2eb39eb6 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:34.529303 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e44d42e-09ec-4983-af83-7c48dbce7d12 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:34.537307 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1332/2648] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:51:34 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 17:51:34.539784 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e44d42e-09ec-4983-af83-7c48dbce7d12 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:34.541443 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b4deee-5784-4d2f-bdab-8fab2eb39eb6 admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:34.544901 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4979e67a-7908-467e-b0ae-581800e2fda6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:34.545500 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4979e67a-7908-467e-b0ae-581800e2fda6 admin admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:34.545779 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4979e67a-7908-467e-b0ae-581800e2fda6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:34.546212 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4979e67a-7908-467e-b0ae-581800e2fda6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:34.546701 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4979e67a-7908-467e-b0ae-581800e2fda6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:34.548300 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4979e67a-7908-467e-b0ae-581800e2fda6 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:34.551418 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b4deee-5784-4d2f-bdab-8fab2eb39eb6 admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:34.556260 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9485c06-3ccd-472f-bfe2-6fe3ddd5fec0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:34.556938 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9485c06-3ccd-472f-bfe2-6fe3ddd5fec0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:34.557334 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9485c06-3ccd-472f-bfe2-6fe3ddd5fec0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:34.557622 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9485c06-3ccd-472f-bfe2-6fe3ddd5fec0 admin admin] PATH_INFO: `/v3/projects/a422d3f56c9f407a9c4a9c35f749f457/users/3edc787356d54127bfd370c9bc6f976b/roles/02476a5b82eb4b188c18a5d706a3ef68` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:34.561632 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1317/2649] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:51:34 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 17:51:34.562594 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4979e67a-7908-467e-b0ae-581800e2fda6 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:34.572594 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4979e67a-7908-467e-b0ae-581800e2fda6 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:34.572933 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9485c06-3ccd-472f-bfe2-6fe3ddd5fec0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a422d3f56c9f407a9c4a9c35f749f457, user_id=3edc787356d54127bfd370c9bc6f976b, role_id=02476a5b82eb4b188c18a5d706a3ef68)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:34.575610 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9485c06-3ccd-472f-bfe2-6fe3ddd5fec0 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:34.577346 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5704d6e-42ee-4a4f-b0db-7d19cac246e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:34.577927 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5704d6e-42ee-4a4f-b0db-7d19cac246e7 admin admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:34.578206 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5704d6e-42ee-4a4f-b0db-7d19cac246e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:34.578476 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5704d6e-42ee-4a4f-b0db-7d19cac246e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:34.579127 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5704d6e-42ee-4a4f-b0db-7d19cac246e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:34.580732 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5704d6e-42ee-4a4f-b0db-7d19cac246e7 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:34.589680 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1333/2650] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 17:51:34 2022] PUT /identity/v3/projects/a422d3f56c9f407a9c4a9c35f749f457/users/3edc787356d54127bfd370c9bc6f976b/roles/02476a5b82eb4b188c18a5d706a3ef68 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:51:34.592463 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1318/2651] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:51:34 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:51:34.596458 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9485c06-3ccd-472f-bfe2-6fe3ddd5fec0 admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:34.601001 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbca2a3-b0f8-42c9-bcd3-8e003846e174 None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:34.601144 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:51:34.601587 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbca2a3-b0f8-42c9-bcd3-8e003846e174 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:34.602086 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbca2a3-b0f8-42c9-bcd3-8e003846e174 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:34.631185 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-edbca2a3-b0f8-42c9-bcd3-8e003846e174 None None] MFA Rules not processed for user `3edc787356d54127bfd370c9bc6f976b`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:51:34.632418 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-edbca2a3-b0f8-42c9-bcd3-8e003846e174 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:34.658687 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9485c06-3ccd-472f-bfe2-6fe3ddd5fec0 admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:34.663824 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8283d65a-c9dc-49fe-8a67-d26af56544c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:34.664490 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8283d65a-c9dc-49fe-8a67-d26af56544c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:34.664806 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8283d65a-c9dc-49fe-8a67-d26af56544c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:34.665089 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8283d65a-c9dc-49fe-8a67-d26af56544c3 admin admin] PATH_INFO: `/v3/projects/dd538bc671184245a0eb9e603d081104/users/85141f5bc82a41b9ab19c9e592f9774d/roles/02476a5b82eb4b188c18a5d706a3ef68` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:34.679612 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8283d65a-c9dc-49fe-8a67-d26af56544c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd538bc671184245a0eb9e603d081104, user_id=85141f5bc82a41b9ab19c9e592f9774d, role_id=02476a5b82eb4b188c18a5d706a3ef68)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:34.682304 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8283d65a-c9dc-49fe-8a67-d26af56544c3 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:34.695761 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1334/2652] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 17:51:34 2022] PUT /identity/v3/projects/dd538bc671184245a0eb9e603d081104/users/85141f5bc82a41b9ab19c9e592f9774d/roles/02476a5b82eb4b188c18a5d706a3ef68 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:51:34.696867 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1319/2653] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:51:34 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:51:34.705869 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eec2a80-d26e-41b9-94b4-1330382febc8 None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:34.706667 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eec2a80-d26e-41b9-94b4-1330382febc8 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:34.706667 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eec2a80-d26e-41b9-94b4-1330382febc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:34.711680 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbca2a3-b0f8-42c9-bcd3-8e003846e174 None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:34.713129 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-edbca2a3-b0f8-42c9-bcd3-8e003846e174 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:34.734902 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-6eec2a80-d26e-41b9-94b4-1330382febc8 None None] MFA Rules not processed for user `85141f5bc82a41b9ab19c9e592f9774d`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:51:34.736117 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-6eec2a80-d26e-41b9-94b4-1330382febc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:34.757256 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:51:34.773180 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbca2a3-b0f8-42c9-bcd3-8e003846e174 None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:34.811884 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1335/2654] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:51:34 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:51:34.823824 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eec2a80-d26e-41b9-94b4-1330382febc8 None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:34.824073 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c596231-12fa-4966-aa85-564bd11ebd3c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59be342541264942ad87f60ff4de6c8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:34.824734 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c596231-12fa-4966-aa85-564bd11ebd3c service cinder] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:34.825022 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c596231-12fa-4966-aa85-564bd11ebd3c service cinder] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:34.825235 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-6eec2a80-d26e-41b9-94b4-1330382febc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:34.825344 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c596231-12fa-4966-aa85-564bd11ebd3c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:34.880856 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eec2a80-d26e-41b9-94b4-1330382febc8 None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:34.881068 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c596231-12fa-4966-aa85-564bd11ebd3c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:34.882490 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c596231-12fa-4966-aa85-564bd11ebd3c service cinder] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:34.884675 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ca8bc4-d11f-40f1-9596-2e0b00f445da service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df5fee57539447428dd1c72eab3cfd9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:34.885287 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ca8bc4-d11f-40f1-9596-2e0b00f445da service glance] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:34.885571 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ca8bc4-d11f-40f1-9596-2e0b00f445da service glance] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:34.885852 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ca8bc4-d11f-40f1-9596-2e0b00f445da service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:34.891810 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1320/2655] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:51:34 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 17:51:34.894227 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ca8bc4-d11f-40f1-9596-2e0b00f445da service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:34.895768 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ca8bc4-d11f-40f1-9596-2e0b00f445da service glance] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:34.905050 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1336/2656] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:51:34 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 17:51:44.887597 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=51 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:51:44.900863 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:51:45.031004 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c596231-12fa-4966-aa85-564bd11ebd3c service cinder] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:45.094845 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c596231-12fa-4966-aa85-564bd11ebd3c service cinder] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:45.099914 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f923c2e-adc3-4284-94a5-c1c41c83d669 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:45.100542 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f923c2e-adc3-4284-94a5-c1c41c83d669 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:45.100850 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f923c2e-adc3-4284-94a5-c1c41c83d669 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:45.101126 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f923c2e-adc3-4284-94a5-c1c41c83d669 admin admin] PATH_INFO: `/v3/users/3edc787356d54127bfd370c9bc6f976b` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:45.102435 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f923c2e-adc3-4284-94a5-c1c41c83d669 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3edc787356d54127bfd370c9bc6f976b)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:45.104372 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f923c2e-adc3-4284-94a5-c1c41c83d669 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:45.150828 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-5f923c2e-adc3-4284-94a5-c1c41c83d669 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3edc787356d54127bfd370c9bc6f976b'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:45.151476 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-5f923c2e-adc3-4284-94a5-c1c41c83d669 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3edc787356d54127bfd370c9bc6f976b'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:45.158887 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-5f923c2e-adc3-4284-94a5-c1c41c83d669 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3edc787356d54127bfd370c9bc6f976b'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:45.160207 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-5f923c2e-adc3-4284-94a5-c1c41c83d669 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3edc787356d54127bfd370c9bc6f976b'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:45.168572 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-5f923c2e-adc3-4284-94a5-c1c41c83d669 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3edc787356d54127bfd370c9bc6f976b'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:45.185767 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1321/2657] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 17:51:45 2022] DELETE /identity/v3/users/3edc787356d54127bfd370c9bc6f976b => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:51:45.190243 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f053b98f-599a-47d3-b801-5544e5f68b01 None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:45.190530 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f053b98f-599a-47d3-b801-5544e5f68b01 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:45.190805 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f053b98f-599a-47d3-b801-5544e5f68b01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:45.209988 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-f053b98f-599a-47d3-b801-5544e5f68b01 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:51:45.211195 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-f053b98f-599a-47d3-b801-5544e5f68b01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:45.273168 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1337/2658] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:51:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:51:45.283079 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f923c2e-adc3-4284-94a5-c1c41c83d669 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:45.284566 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-5f923c2e-adc3-4284-94a5-c1c41c83d669 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:45.342492 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f923c2e-adc3-4284-94a5-c1c41c83d669 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:45.346227 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62040fc-9aec-4b04-a5e3-cd7eadd67cdd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43ad98a250a144099ccd18378aedd4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:45.346773 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62040fc-9aec-4b04-a5e3-cd7eadd67cdd service neutron] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:45.347551 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62040fc-9aec-4b04-a5e3-cd7eadd67cdd service neutron] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:45.347823 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62040fc-9aec-4b04-a5e3-cd7eadd67cdd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:45.356704 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62040fc-9aec-4b04-a5e3-cd7eadd67cdd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:45.358134 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62040fc-9aec-4b04-a5e3-cd7eadd67cdd service neutron] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:45.368653 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1322/2659] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:51:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:51:45.637144 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f053b98f-599a-47d3-b801-5544e5f68b01 None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:45.638544 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-f053b98f-599a-47d3-b801-5544e5f68b01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:45.652202 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f053b98f-599a-47d3-b801-5544e5f68b01 None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:45.657169 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed57bdc-db86-4a86-96a1-996e397f7eaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:45.657843 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed57bdc-db86-4a86-96a1-996e397f7eaa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:45.658119 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed57bdc-db86-4a86-96a1-996e397f7eaa admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:45.658391 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed57bdc-db86-4a86-96a1-996e397f7eaa admin admin] PATH_INFO: `/v3/projects/a422d3f56c9f407a9c4a9c35f749f457` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:45.660840 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed57bdc-db86-4a86-96a1-996e397f7eaa admin admin] RBAC: Authorizing `identity:delete_project(project_id=a422d3f56c9f407a9c4a9c35f749f457)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:45.662380 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed57bdc-db86-4a86-96a1-996e397f7eaa admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:45.832876 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-2ed57bdc-db86-4a86-96a1-996e397f7eaa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a422d3f56c9f407a9c4a9c35f749f457', 'request_id': 'req-2ed57bdc-db86-4a86-96a1-996e397f7eaa'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:45.840505 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-2ed57bdc-db86-4a86-96a1-996e397f7eaa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a422d3f56c9f407a9c4a9c35f749f457', 'request_id': 'req-2ed57bdc-db86-4a86-96a1-996e397f7eaa'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:45.847206 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-2ed57bdc-db86-4a86-96a1-996e397f7eaa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a422d3f56c9f407a9c4a9c35f749f457', 'request_id': 'req-2ed57bdc-db86-4a86-96a1-996e397f7eaa'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:45.858523 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-2ed57bdc-db86-4a86-96a1-996e397f7eaa admin admin] The token cache is being invalidate because project a422d3f56c9f407a9c4a9c35f749f457 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81758) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:51:45.858626 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-2ed57bdc-db86-4a86-96a1-996e397f7eaa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:45.859226 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-2ed57bdc-db86-4a86-96a1-996e397f7eaa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:51:45.861442 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1338/2660] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 17:51:45 2022] DELETE /identity/v3/projects/a422d3f56c9f407a9c4a9c35f749f457 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:51:55.364141 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:51:55.855387 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:52:00.916155 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62040fc-9aec-4b04-a5e3-cd7eadd67cdd service neutron] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:52:00.917682 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-c62040fc-9aec-4b04-a5e3-cd7eadd67cdd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:52:00.930348 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62040fc-9aec-4b04-a5e3-cd7eadd67cdd service neutron] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:52:00.985654 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54743d6-4d36-4bb7-84b3-e8abd284a895 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:52:00.986276 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54743d6-4d36-4bb7-84b3-e8abd284a895 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:52:00.986556 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54743d6-4d36-4bb7-84b3-e8abd284a895 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:52:00.986831 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54743d6-4d36-4bb7-84b3-e8abd284a895 admin admin] PATH_INFO: `/v3/users/85141f5bc82a41b9ab19c9e592f9774d` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:52:00.987966 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54743d6-4d36-4bb7-84b3-e8abd284a895 admin admin] RBAC: Authorizing `identity:delete_user(user_id=85141f5bc82a41b9ab19c9e592f9774d)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:52:00.989920 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54743d6-4d36-4bb7-84b3-e8abd284a895 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:52:01.037234 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-f54743d6-4d36-4bb7-84b3-e8abd284a895 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '85141f5bc82a41b9ab19c9e592f9774d'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:52:01.037874 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-f54743d6-4d36-4bb7-84b3-e8abd284a895 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '85141f5bc82a41b9ab19c9e592f9774d'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:52:01.044674 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-f54743d6-4d36-4bb7-84b3-e8abd284a895 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '85141f5bc82a41b9ab19c9e592f9774d'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:52:01.045361 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-f54743d6-4d36-4bb7-84b3-e8abd284a895 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '85141f5bc82a41b9ab19c9e592f9774d'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:52:01.053232 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-f54743d6-4d36-4bb7-84b3-e8abd284a895 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '85141f5bc82a41b9ab19c9e592f9774d'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:52:01.070650 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1323/2661] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 17:52:00 2022] DELETE /identity/v3/users/85141f5bc82a41b9ab19c9e592f9774d => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:52:01.075208 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773ed4a8-d92b-459e-af0a-8ec8d3b5c199 None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:52:01.075699 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773ed4a8-d92b-459e-af0a-8ec8d3b5c199 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:52:01.075804 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773ed4a8-d92b-459e-af0a-8ec8d3b5c199 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:52:01.093787 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-773ed4a8-d92b-459e-af0a-8ec8d3b5c199 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:52:01.094983 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-773ed4a8-d92b-459e-af0a-8ec8d3b5c199 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:52:01.151687 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1339/2662] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:52:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:52:01.161910 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54743d6-4d36-4bb7-84b3-e8abd284a895 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:52:01.163436 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-f54743d6-4d36-4bb7-84b3-e8abd284a895 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:52:01.219683 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54743d6-4d36-4bb7-84b3-e8abd284a895 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:52:01.223844 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7573af-6a57-4161-b087-4074e2925cb1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43ad98a250a144099ccd18378aedd4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:52:01.224437 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7573af-6a57-4161-b087-4074e2925cb1 service neutron] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:52:01.224748 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7573af-6a57-4161-b087-4074e2925cb1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:52:01.225023 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7573af-6a57-4161-b087-4074e2925cb1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:52:01.233727 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7573af-6a57-4161-b087-4074e2925cb1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:52:01.235141 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7573af-6a57-4161-b087-4074e2925cb1 service neutron] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:52:01.244949 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1324/2663] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:52:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:52:01.508240 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773ed4a8-d92b-459e-af0a-8ec8d3b5c199 None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:52:01.509735 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-773ed4a8-d92b-459e-af0a-8ec8d3b5c199 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:52:01.528012 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773ed4a8-d92b-459e-af0a-8ec8d3b5c199 None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:52:01.534201 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac2d64d-15af-4ad7-a081-368481637002 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:52:01.534826 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac2d64d-15af-4ad7-a081-368481637002 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:52:01.535104 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac2d64d-15af-4ad7-a081-368481637002 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:52:01.535413 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac2d64d-15af-4ad7-a081-368481637002 admin admin] PATH_INFO: `/v3/projects/dd538bc671184245a0eb9e603d081104` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:52:01.536438 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac2d64d-15af-4ad7-a081-368481637002 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dd538bc671184245a0eb9e603d081104)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:52:01.538353 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac2d64d-15af-4ad7-a081-368481637002 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:52:01.582499 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-bac2d64d-15af-4ad7-a081-368481637002 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dd538bc671184245a0eb9e603d081104', 'request_id': 'req-bac2d64d-15af-4ad7-a081-368481637002'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:52:01.589262 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-bac2d64d-15af-4ad7-a081-368481637002 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dd538bc671184245a0eb9e603d081104', 'request_id': 'req-bac2d64d-15af-4ad7-a081-368481637002'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:52:01.596085 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-bac2d64d-15af-4ad7-a081-368481637002 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dd538bc671184245a0eb9e603d081104', 'request_id': 'req-bac2d64d-15af-4ad7-a081-368481637002'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:52:01.606388 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-bac2d64d-15af-4ad7-a081-368481637002 admin admin] The token cache is being invalidate because project dd538bc671184245a0eb9e603d081104 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81758) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:52:01.606678 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-bac2d64d-15af-4ad7-a081-368481637002 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:52:01.607303 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-bac2d64d-15af-4ad7-a081-368481637002 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:52:01.608784 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1340/2664] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 17:52:01 2022] DELETE /identity/v3/projects/dd538bc671184245a0eb9e603d081104 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:52:01.614388 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40203336-841d-4994-8091-9a4ea34f37bb None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:52:01.614674 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40203336-841d-4994-8091-9a4ea34f37bb None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:52:01.614948 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40203336-841d-4994-8091-9a4ea34f37bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:52:01.634876 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-40203336-841d-4994-8091-9a4ea34f37bb None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:52:01.636060 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-40203336-841d-4994-8091-9a4ea34f37bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:52:01.653312 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:52:01.695789 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1325/2665] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:52:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:52:01.705588 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a07bebd-ee2d-4367-a548-bcf403d7511c None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:52:01.705876 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a07bebd-ee2d-4367-a548-bcf403d7511c None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:52:01.706154 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a07bebd-ee2d-4367-a548-bcf403d7511c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:52:01.724983 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-3a07bebd-ee2d-4367-a548-bcf403d7511c None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:52:01.726194 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-3a07bebd-ee2d-4367-a548-bcf403d7511c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:52:01.733984 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1341/2666] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:52:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:52:01.738059 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40203336-841d-4994-8091-9a4ea34f37bb None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:52:01.747593 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40203336-841d-4994-8091-9a4ea34f37bb None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:52:01.752805 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a1f04f-b25e-40ea-bfb6-8c3992f0c1e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:52:01.753396 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a1f04f-b25e-40ea-bfb6-8c3992f0c1e2 admin admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:52:01.753692 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a1f04f-b25e-40ea-bfb6-8c3992f0c1e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:52:01.753949 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a1f04f-b25e-40ea-bfb6-8c3992f0c1e2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:52:01.754591 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a1f04f-b25e-40ea-bfb6-8c3992f0c1e2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:52:01.754894 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a1f04f-b25e-40ea-bfb6-8c3992f0c1e2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:52:01.756481 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a1f04f-b25e-40ea-bfb6-8c3992f0c1e2 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:52:01.768650 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1326/2667] 10.222.0.45 () {58 vars in 1153 bytes} [Wed Aug 24 17:52:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:52:01.773100 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a07bebd-ee2d-4367-a548-bcf403d7511c None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:52:01.782398 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a07bebd-ee2d-4367-a548-bcf403d7511c None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:52:01.787186 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d172881-9d49-435e-87d5-1ca7271b92da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:52:01.787784 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d172881-9d49-435e-87d5-1ca7271b92da admin admin] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:52:01.788063 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d172881-9d49-435e-87d5-1ca7271b92da admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:52:01.788336 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d172881-9d49-435e-87d5-1ca7271b92da admin admin] PATH_INFO: `/v3/projects` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:52:01.789002 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d172881-9d49-435e-87d5-1ca7271b92da admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:52:01.790879 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d172881-9d49-435e-87d5-1ca7271b92da admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:52:01.818161 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1342/2668] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 17:52:01 2022] POST /identity/v3/projects => generated 361 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:52:01.822384 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a1f04f-b25e-40ea-bfb6-8c3992f0c1e2 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:52:01.833214 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a1f04f-b25e-40ea-bfb6-8c3992f0c1e2 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:52:01.887886 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67da3f51-8bc3-4382-8cd9-438cfbf16f37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:52:01.888414 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67da3f51-8bc3-4382-8cd9-438cfbf16f37 admin admin] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:52:01.888732 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67da3f51-8bc3-4382-8cd9-438cfbf16f37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:52:01.889008 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67da3f51-8bc3-4382-8cd9-438cfbf16f37 admin admin] PATH_INFO: `/v3/users` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:52:01.889638 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67da3f51-8bc3-4382-8cd9-438cfbf16f37 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:52:01.891539 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67da3f51-8bc3-4382-8cd9-438cfbf16f37 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:52:01.912631 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1327/2669] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 17:52:01 2022] POST /identity/v3/users => generated 281 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:52:01.916630 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d172881-9d49-435e-87d5-1ca7271b92da admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:52:01.926630 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d172881-9d49-435e-87d5-1ca7271b92da admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:52:01.931375 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a52c63-1183-42ff-a99b-8e4c16818597 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:52:01.931978 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a52c63-1183-42ff-a99b-8e4c16818597 admin admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:52:01.932259 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a52c63-1183-42ff-a99b-8e4c16818597 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:52:01.932535 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a52c63-1183-42ff-a99b-8e4c16818597 admin admin] PATH_INFO: `/v3/roles` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:52:01.933241 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a52c63-1183-42ff-a99b-8e4c16818597 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:52:01.934853 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a52c63-1183-42ff-a99b-8e4c16818597 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:52:01.945285 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1343/2670] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:52:01 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:52:01.948957 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67da3f51-8bc3-4382-8cd9-438cfbf16f37 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:52:01.959517 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67da3f51-8bc3-4382-8cd9-438cfbf16f37 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:52:01.964270 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee02696-64f3-4905-9929-7249e9e9c9aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:52:01.964904 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee02696-64f3-4905-9929-7249e9e9c9aa admin admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:52:01.965185 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee02696-64f3-4905-9929-7249e9e9c9aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:52:01.965465 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee02696-64f3-4905-9929-7249e9e9c9aa admin admin] PATH_INFO: `/v3/roles` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:52:01.966227 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee02696-64f3-4905-9929-7249e9e9c9aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:52:01.967759 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee02696-64f3-4905-9929-7249e9e9c9aa admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:52:01.978494 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1328/2671] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:52:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:52:01.982668 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a52c63-1183-42ff-a99b-8e4c16818597 admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:52:01.992558 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a52c63-1183-42ff-a99b-8e4c16818597 admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:52:01.997087 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0932ac-ff65-4868-9865-7b3932bc387b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:52:01.997724 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0932ac-ff65-4868-9865-7b3932bc387b admin admin] REQUEST_METHOD: `PUT` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:52:01.998004 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0932ac-ff65-4868-9865-7b3932bc387b admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:52:01.998284 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0932ac-ff65-4868-9865-7b3932bc387b admin admin] PATH_INFO: `/v3/projects/08b77f33d8df468f8572e1ba3e994ff1/users/ec3fb05983a948e086bad343da35811a/roles/02476a5b82eb4b188c18a5d706a3ef68` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:52:02.012923 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0932ac-ff65-4868-9865-7b3932bc387b admin admin] RBAC: Authorizing `identity:create_grant(project_id=08b77f33d8df468f8572e1ba3e994ff1, user_id=ec3fb05983a948e086bad343da35811a, role_id=02476a5b82eb4b188c18a5d706a3ef68)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:52:02.015622 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0932ac-ff65-4868-9865-7b3932bc387b admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:52:02.028609 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1344/2672] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 17:52:01 2022] PUT /identity/v3/projects/08b77f33d8df468f8572e1ba3e994ff1/users/ec3fb05983a948e086bad343da35811a/roles/02476a5b82eb4b188c18a5d706a3ef68 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:52:02.039083 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2928f676-1718-4d0b-bc91-9aa2fe0c67f0 None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:52:02.039371 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2928f676-1718-4d0b-bc91-9aa2fe0c67f0 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:52:02.039645 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2928f676-1718-4d0b-bc91-9aa2fe0c67f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:52:02.067335 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-2928f676-1718-4d0b-bc91-9aa2fe0c67f0 None None] MFA Rules not processed for user `ec3fb05983a948e086bad343da35811a`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:52:02.068542 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-2928f676-1718-4d0b-bc91-9aa2fe0c67f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:52:02.130467 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1329/2673] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:52:02 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:52:02.141401 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0932ac-ff65-4868-9865-7b3932bc387b admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:52:02.143727 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-2c0932ac-ff65-4868-9865-7b3932bc387b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:52:02.165725 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:52:02.200455 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0932ac-ff65-4868-9865-7b3932bc387b admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:52:02.204069 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018b127b-d05d-4cdb-b611-14d855f99f54 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59be342541264942ad87f60ff4de6c8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:52:02.205228 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018b127b-d05d-4cdb-b611-14d855f99f54 service cinder] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:52:02.205511 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018b127b-d05d-4cdb-b611-14d855f99f54 service cinder] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:52:02.205786 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018b127b-d05d-4cdb-b611-14d855f99f54 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:52:02.213844 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018b127b-d05d-4cdb-b611-14d855f99f54 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:52:02.215262 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018b127b-d05d-4cdb-b611-14d855f99f54 service cinder] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:52:02.224394 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1345/2674] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:52:02 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:52:12.127578 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=89,keystone:INSERT=3 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:52:12.220400 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=17 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:52:40.701015 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2928f676-1718-4d0b-bc91-9aa2fe0c67f0 None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:52:40.702541 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-2928f676-1718-4d0b-bc91-9aa2fe0c67f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:52:40.716609 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2928f676-1718-4d0b-bc91-9aa2fe0c67f0 None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:52:40.720195 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f118ec85-cad2-4746-929b-8edb82f6b1fc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59be342541264942ad87f60ff4de6c8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:52:40.720835 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f118ec85-cad2-4746-929b-8edb82f6b1fc service cinder] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:52:40.721183 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f118ec85-cad2-4746-929b-8edb82f6b1fc service cinder] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:52:40.721454 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f118ec85-cad2-4746-929b-8edb82f6b1fc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:52:40.723703 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-f118ec85-cad2-4746-929b-8edb82f6b1fc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:52:40.781942 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f118ec85-cad2-4746-929b-8edb82f6b1fc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:52:40.783622 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f118ec85-cad2-4746-929b-8edb82f6b1fc service cinder] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:52:40.793113 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1330/2675] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:52:40 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:52:44.573805 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018b127b-d05d-4cdb-b611-14d855f99f54 service cinder] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:52:44.634559 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018b127b-d05d-4cdb-b611-14d855f99f54 service cinder] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:52:44.639094 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326ab57d-cee0-4f4a-976d-80c5970488e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:52:44.639714 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326ab57d-cee0-4f4a-976d-80c5970488e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:52:44.639995 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326ab57d-cee0-4f4a-976d-80c5970488e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:52:44.640272 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326ab57d-cee0-4f4a-976d-80c5970488e3 admin admin] PATH_INFO: `/v3/users/ec3fb05983a948e086bad343da35811a` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:52:44.641416 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326ab57d-cee0-4f4a-976d-80c5970488e3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ec3fb05983a948e086bad343da35811a)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:52:44.643367 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326ab57d-cee0-4f4a-976d-80c5970488e3 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:52:44.687217 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-326ab57d-cee0-4f4a-976d-80c5970488e3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ec3fb05983a948e086bad343da35811a'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:52:44.687811 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-326ab57d-cee0-4f4a-976d-80c5970488e3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ec3fb05983a948e086bad343da35811a'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:52:44.695136 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-326ab57d-cee0-4f4a-976d-80c5970488e3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ec3fb05983a948e086bad343da35811a'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:52:44.696560 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-326ab57d-cee0-4f4a-976d-80c5970488e3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ec3fb05983a948e086bad343da35811a'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:52:44.708649 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-326ab57d-cee0-4f4a-976d-80c5970488e3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ec3fb05983a948e086bad343da35811a'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:52:44.727485 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1346/2676] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 17:52:44 2022] DELETE /identity/v3/users/ec3fb05983a948e086bad343da35811a => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:52:44.734068 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bd774c-b71b-4bab-8042-a09b4669c2cd None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:52:44.734068 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bd774c-b71b-4bab-8042-a09b4669c2cd None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:52:44.736297 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bd774c-b71b-4bab-8042-a09b4669c2cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:52:44.758183 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-62bd774c-b71b-4bab-8042-a09b4669c2cd None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:52:44.759587 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-62bd774c-b71b-4bab-8042-a09b4669c2cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:52:44.826561 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1331/2677] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:52:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:52:44.836229 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326ab57d-cee0-4f4a-976d-80c5970488e3 admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:52:44.837730 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-326ab57d-cee0-4f4a-976d-80c5970488e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:52:44.896109 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326ab57d-cee0-4f4a-976d-80c5970488e3 admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:52:44.900091 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13e5902-4346-4e80-af0e-8df0f109acda service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43ad98a250a144099ccd18378aedd4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:52:44.900741 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13e5902-4346-4e80-af0e-8df0f109acda service neutron] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:52:44.901156 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13e5902-4346-4e80-af0e-8df0f109acda service neutron] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:52:44.901498 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13e5902-4346-4e80-af0e-8df0f109acda service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:52:44.911768 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13e5902-4346-4e80-af0e-8df0f109acda service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:52:44.913249 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13e5902-4346-4e80-af0e-8df0f109acda service neutron] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:52:44.923458 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1347/2678] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:52:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:52:45.328414 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bd774c-b71b-4bab-8042-a09b4669c2cd None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:52:45.329933 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-62bd774c-b71b-4bab-8042-a09b4669c2cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:52:45.342223 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bd774c-b71b-4bab-8042-a09b4669c2cd None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:52:45.347432 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6703d8-e5f4-41a7-acc8-8dbb4d390181 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:52:45.348055 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6703d8-e5f4-41a7-acc8-8dbb4d390181 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:52:45.348333 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6703d8-e5f4-41a7-acc8-8dbb4d390181 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:52:45.348636 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6703d8-e5f4-41a7-acc8-8dbb4d390181 admin admin] PATH_INFO: `/v3/projects/08b77f33d8df468f8572e1ba3e994ff1` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:52:45.349754 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6703d8-e5f4-41a7-acc8-8dbb4d390181 admin admin] RBAC: Authorizing `identity:delete_project(project_id=08b77f33d8df468f8572e1ba3e994ff1)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:52:45.351664 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6703d8-e5f4-41a7-acc8-8dbb4d390181 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:52:45.407774 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-3e6703d8-e5f4-41a7-acc8-8dbb4d390181 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '08b77f33d8df468f8572e1ba3e994ff1', 'request_id': 'req-3e6703d8-e5f4-41a7-acc8-8dbb4d390181'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:52:45.416729 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-3e6703d8-e5f4-41a7-acc8-8dbb4d390181 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '08b77f33d8df468f8572e1ba3e994ff1', 'request_id': 'req-3e6703d8-e5f4-41a7-acc8-8dbb4d390181'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:52:45.423184 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-3e6703d8-e5f4-41a7-acc8-8dbb4d390181 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '08b77f33d8df468f8572e1ba3e994ff1', 'request_id': 'req-3e6703d8-e5f4-41a7-acc8-8dbb4d390181'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:52:45.436307 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-3e6703d8-e5f4-41a7-acc8-8dbb4d390181 admin admin] The token cache is being invalidate because project 08b77f33d8df468f8572e1ba3e994ff1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81759) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:52:45.436631 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-3e6703d8-e5f4-41a7-acc8-8dbb4d390181 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:52:45.437244 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-3e6703d8-e5f4-41a7-acc8-8dbb4d390181 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:52:45.438909 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1332/2679] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 17:52:45 2022] DELETE /identity/v3/projects/08b77f33d8df468f8572e1ba3e994ff1 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:52:45.443670 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a46723-73e4-45c0-b799-349e066a7156 None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:52:45.443952 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a46723-73e4-45c0-b799-349e066a7156 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:52:45.444223 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a46723-73e4-45c0-b799-349e066a7156 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:52:45.465114 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-05a46723-73e4-45c0-b799-349e066a7156 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:52:45.467315 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-05a46723-73e4-45c0-b799-349e066a7156 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:52:45.486613 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:52:45.532770 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1348/2680] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:52:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:52:45.543379 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d9a27b-d258-4392-b6ce-5006d4715b90 None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:52:45.543659 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d9a27b-d258-4392-b6ce-5006d4715b90 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:52:45.543933 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d9a27b-d258-4392-b6ce-5006d4715b90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:52:45.564470 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-d9d9a27b-d258-4392-b6ce-5006d4715b90 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:52:45.565749 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-d9d9a27b-d258-4392-b6ce-5006d4715b90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:52:45.573838 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1333/2681] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:52:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:52:45.578750 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a46723-73e4-45c0-b799-349e066a7156 None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:52:45.589290 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a46723-73e4-45c0-b799-349e066a7156 None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:52:45.594302 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d41c41-ea94-4ba6-ac75-23434447cfcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:52:45.594903 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d41c41-ea94-4ba6-ac75-23434447cfcd admin admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:52:45.595183 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d41c41-ea94-4ba6-ac75-23434447cfcd admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:52:45.595439 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d41c41-ea94-4ba6-ac75-23434447cfcd admin admin] PATH_INFO: `/v3/domains` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:52:45.596141 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d41c41-ea94-4ba6-ac75-23434447cfcd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:52:45.596444 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d41c41-ea94-4ba6-ac75-23434447cfcd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:52:45.598086 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d41c41-ea94-4ba6-ac75-23434447cfcd admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:52:45.610090 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1349/2682] 10.222.0.45 () {58 vars in 1153 bytes} [Wed Aug 24 17:52:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:52:45.615174 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d9a27b-d258-4392-b6ce-5006d4715b90 None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:52:45.624889 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d9a27b-d258-4392-b6ce-5006d4715b90 None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:52:45.630678 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e64e8d5-68cf-4108-8ba2-de8c31e4566d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:52:45.631707 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e64e8d5-68cf-4108-8ba2-de8c31e4566d admin admin] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:52:45.631707 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e64e8d5-68cf-4108-8ba2-de8c31e4566d admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:52:45.631930 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e64e8d5-68cf-4108-8ba2-de8c31e4566d admin admin] PATH_INFO: `/v3/projects` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:52:45.632573 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e64e8d5-68cf-4108-8ba2-de8c31e4566d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:52:45.634484 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e64e8d5-68cf-4108-8ba2-de8c31e4566d admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:52:45.649824 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:52:45.665013 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1334/2683] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 17:52:45 2022] POST /identity/v3/projects => generated 373 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:52:45.669142 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d41c41-ea94-4ba6-ac75-23434447cfcd admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:52:45.681769 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d41c41-ea94-4ba6-ac75-23434447cfcd admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:52:45.749817 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1238d2ef-ae91-4307-a650-4c977f9e4600 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:52:45.750419 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1238d2ef-ae91-4307-a650-4c977f9e4600 admin admin] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:52:45.750753 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1238d2ef-ae91-4307-a650-4c977f9e4600 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:52:45.751026 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1238d2ef-ae91-4307-a650-4c977f9e4600 admin admin] PATH_INFO: `/v3/users` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:52:45.751659 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1238d2ef-ae91-4307-a650-4c977f9e4600 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:52:45.754236 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1238d2ef-ae91-4307-a650-4c977f9e4600 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:52:45.781583 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1350/2684] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 17:52:45 2022] POST /identity/v3/users => generated 287 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:52:45.786481 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e64e8d5-68cf-4108-8ba2-de8c31e4566d admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:52:45.798595 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e64e8d5-68cf-4108-8ba2-de8c31e4566d admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:52:45.805301 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a68d2c-5001-4686-9bd8-d74247ec9714 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:52:45.805878 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a68d2c-5001-4686-9bd8-d74247ec9714 admin admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:52:45.806162 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a68d2c-5001-4686-9bd8-d74247ec9714 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:52:45.806448 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a68d2c-5001-4686-9bd8-d74247ec9714 admin admin] PATH_INFO: `/v3/roles` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:52:45.807204 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a68d2c-5001-4686-9bd8-d74247ec9714 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:52:45.809345 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a68d2c-5001-4686-9bd8-d74247ec9714 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:52:45.823540 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1335/2685] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:52:45 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:52:45.824848 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1238d2ef-ae91-4307-a650-4c977f9e4600 admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:52:45.835200 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1238d2ef-ae91-4307-a650-4c977f9e4600 admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:52:45.840142 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe8fd27-8dee-4ffa-a037-5d18a6548573 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:52:45.840803 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe8fd27-8dee-4ffa-a037-5d18a6548573 admin admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:52:45.841088 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe8fd27-8dee-4ffa-a037-5d18a6548573 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:52:45.841358 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe8fd27-8dee-4ffa-a037-5d18a6548573 admin admin] PATH_INFO: `/v3/roles` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:52:45.841999 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe8fd27-8dee-4ffa-a037-5d18a6548573 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:52:45.843575 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe8fd27-8dee-4ffa-a037-5d18a6548573 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:52:45.856940 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1351/2686] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:52:45 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 17:52:45.860922 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a68d2c-5001-4686-9bd8-d74247ec9714 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:52:45.872270 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a68d2c-5001-4686-9bd8-d74247ec9714 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:52:45.876610 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faaec74e-2c26-46c4-b58d-93c354432375 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:52:45.877295 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaec74e-2c26-46c4-b58d-93c354432375 admin admin] REQUEST_METHOD: `PUT` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:52:45.877576 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaec74e-2c26-46c4-b58d-93c354432375 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:52:45.877885 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaec74e-2c26-46c4-b58d-93c354432375 admin admin] PATH_INFO: `/v3/projects/481e19da29b04ced991e8a86fb862a79/users/55a3c456c9c14ec19e4fe2f36893fb63/roles/02476a5b82eb4b188c18a5d706a3ef68` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:52:45.893736 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faaec74e-2c26-46c4-b58d-93c354432375 admin admin] RBAC: Authorizing `identity:create_grant(project_id=481e19da29b04ced991e8a86fb862a79, user_id=55a3c456c9c14ec19e4fe2f36893fb63, role_id=02476a5b82eb4b188c18a5d706a3ef68)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:52:45.896400 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faaec74e-2c26-46c4-b58d-93c354432375 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:52:45.911154 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1336/2687] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 17:52:45 2022] PUT /identity/v3/projects/481e19da29b04ced991e8a86fb862a79/users/55a3c456c9c14ec19e4fe2f36893fb63/roles/02476a5b82eb4b188c18a5d706a3ef68 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:52:45.921643 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8946df-ae98-4448-a2ad-63216bf3c4a6 None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:52:45.921942 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8946df-ae98-4448-a2ad-63216bf3c4a6 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:52:45.922280 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8946df-ae98-4448-a2ad-63216bf3c4a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:52:45.951873 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-fa8946df-ae98-4448-a2ad-63216bf3c4a6 None None] MFA Rules not processed for user `55a3c456c9c14ec19e4fe2f36893fb63`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:52:45.953020 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-fa8946df-ae98-4448-a2ad-63216bf3c4a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:52:46.018152 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1352/2688] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:52:45 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:52:46.029156 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faaec74e-2c26-46c4-b58d-93c354432375 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:52:46.030868 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-faaec74e-2c26-46c4-b58d-93c354432375 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:52:46.093238 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faaec74e-2c26-46c4-b58d-93c354432375 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:52:46.097746 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4383c8fa-cef2-4ee6-8361-3e41735e4086 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59be342541264942ad87f60ff4de6c8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:52:46.098356 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4383c8fa-cef2-4ee6-8361-3e41735e4086 service cinder] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:52:46.098646 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4383c8fa-cef2-4ee6-8361-3e41735e4086 service cinder] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:52:46.098903 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4383c8fa-cef2-4ee6-8361-3e41735e4086 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:52:46.107246 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4383c8fa-cef2-4ee6-8361-3e41735e4086 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:52:46.108894 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4383c8fa-cef2-4ee6-8361-3e41735e4086 service cinder] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:52:46.118916 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1337/2689] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:52:46 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:52:46.143508 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8946df-ae98-4448-a2ad-63216bf3c4a6 None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:52:46.163572 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:52:46.210038 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8946df-ae98-4448-a2ad-63216bf3c4a6 None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:52:46.214634 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c6277c-d753-41f5-a4fe-0998f6a5cd21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:52:46.215270 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c6277c-d753-41f5-a4fe-0998f6a5cd21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:52:46.215547 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c6277c-d753-41f5-a4fe-0998f6a5cd21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:52:46.215817 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c6277c-d753-41f5-a4fe-0998f6a5cd21 admin admin] PATH_INFO: `/v3/users/55a3c456c9c14ec19e4fe2f36893fb63` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:52:46.216977 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c6277c-d753-41f5-a4fe-0998f6a5cd21 admin admin] RBAC: Authorizing `identity:delete_user(user_id=55a3c456c9c14ec19e4fe2f36893fb63)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:52:46.218895 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c6277c-d753-41f5-a4fe-0998f6a5cd21 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:52:46.264423 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-e8c6277c-d753-41f5-a4fe-0998f6a5cd21 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '55a3c456c9c14ec19e4fe2f36893fb63'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:52:46.265180 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-e8c6277c-d753-41f5-a4fe-0998f6a5cd21 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '55a3c456c9c14ec19e4fe2f36893fb63'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:52:46.273395 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-e8c6277c-d753-41f5-a4fe-0998f6a5cd21 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '55a3c456c9c14ec19e4fe2f36893fb63'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:52:46.274038 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-e8c6277c-d753-41f5-a4fe-0998f6a5cd21 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '55a3c456c9c14ec19e4fe2f36893fb63'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:52:46.288209 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-e8c6277c-d753-41f5-a4fe-0998f6a5cd21 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '55a3c456c9c14ec19e4fe2f36893fb63'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:52:46.301903 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1353/2690] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 17:52:46 2022] DELETE /identity/v3/users/55a3c456c9c14ec19e4fe2f36893fb63 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:52:46.307552 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f0c9d9-495b-492f-8b68-9fa09a9cd118 None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:52:46.307842 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f0c9d9-495b-492f-8b68-9fa09a9cd118 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:52:46.308109 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f0c9d9-495b-492f-8b68-9fa09a9cd118 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:52:46.326539 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-a5f0c9d9-495b-492f-8b68-9fa09a9cd118 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:52:46.327743 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-a5f0c9d9-495b-492f-8b68-9fa09a9cd118 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:52:46.392149 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1338/2691] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:52:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:52:46.403220 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c6277c-d753-41f5-a4fe-0998f6a5cd21 admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:52:46.405043 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-e8c6277c-d753-41f5-a4fe-0998f6a5cd21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:52:46.463440 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c6277c-d753-41f5-a4fe-0998f6a5cd21 admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:52:46.467123 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88cf563-17d3-4358-9ce4-d01c19028dee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43ad98a250a144099ccd18378aedd4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:52:46.467737 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88cf563-17d3-4358-9ce4-d01c19028dee service neutron] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:52:46.468019 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88cf563-17d3-4358-9ce4-d01c19028dee service neutron] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:52:46.468293 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88cf563-17d3-4358-9ce4-d01c19028dee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:52:46.477939 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88cf563-17d3-4358-9ce4-d01c19028dee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:52:46.479421 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88cf563-17d3-4358-9ce4-d01c19028dee service neutron] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:52:46.489978 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1354/2692] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:52:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 17:52:46.851946 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2571c51e-fe43-42cf-8ba2-00c713e6f03a None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:52:46.852432 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2571c51e-fe43-42cf-8ba2-00c713e6f03a None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:52:46.852590 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2571c51e-fe43-42cf-8ba2-00c713e6f03a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:52:46.873081 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-2571c51e-fe43-42cf-8ba2-00c713e6f03a None None] MFA Rules not processed for user `58652b0d60ec4134be18dafeb7ad5cb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:52:46.874293 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-2571c51e-fe43-42cf-8ba2-00c713e6f03a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:52:46.931753 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1339/2693] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:52:46 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:52:46.954060 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88cf563-17d3-4358-9ce4-d01c19028dee service neutron] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:52:46.955488 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-a88cf563-17d3-4358-9ce4-d01c19028dee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:52:46.969725 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88cf563-17d3-4358-9ce4-d01c19028dee service neutron] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:52:46.974581 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4b0568-3311-445c-9d36-5aa2b0aba755 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:52:46.975297 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4b0568-3311-445c-9d36-5aa2b0aba755 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:52:46.975994 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4b0568-3311-445c-9d36-5aa2b0aba755 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:52:46.975994 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4b0568-3311-445c-9d36-5aa2b0aba755 admin admin] PATH_INFO: `/v3/projects/481e19da29b04ced991e8a86fb862a79` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:52:46.977160 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4b0568-3311-445c-9d36-5aa2b0aba755 admin admin] RBAC: Authorizing `identity:delete_project(project_id=481e19da29b04ced991e8a86fb862a79)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:52:46.979279 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4b0568-3311-445c-9d36-5aa2b0aba755 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:52:47.021995 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:52:47.034843 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-2a4b0568-3311-445c-9d36-5aa2b0aba755 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '481e19da29b04ced991e8a86fb862a79', 'request_id': 'req-2a4b0568-3311-445c-9d36-5aa2b0aba755'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:52:47.043241 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-2a4b0568-3311-445c-9d36-5aa2b0aba755 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '481e19da29b04ced991e8a86fb862a79', 'request_id': 'req-2a4b0568-3311-445c-9d36-5aa2b0aba755'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:52:47.050266 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-2a4b0568-3311-445c-9d36-5aa2b0aba755 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '481e19da29b04ced991e8a86fb862a79', 'request_id': 'req-2a4b0568-3311-445c-9d36-5aa2b0aba755'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:52:47.061897 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-2a4b0568-3311-445c-9d36-5aa2b0aba755 admin admin] The token cache is being invalidate because project 481e19da29b04ced991e8a86fb862a79 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81758) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:52:47.062221 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-2a4b0568-3311-445c-9d36-5aa2b0aba755 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:52:47.062872 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-2a4b0568-3311-445c-9d36-5aa2b0aba755 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:52:47.064992 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1355/2694] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 17:52:46 2022] DELETE /identity/v3/projects/481e19da29b04ced991e8a86fb862a79 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:52:47.069793 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae8ba87-cd87-4d5e-ad68-1e765d34c5db None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:52:47.070086 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae8ba87-cd87-4d5e-ad68-1e765d34c5db None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:52:47.070366 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae8ba87-cd87-4d5e-ad68-1e765d34c5db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:52:47.084263 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:52:47.090880 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-7ae8ba87-cd87-4d5e-ad68-1e765d34c5db None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:52:47.092355 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-7ae8ba87-cd87-4d5e-ad68-1e765d34c5db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:52:47.153915 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1340/2695] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:52:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:52:47.161574 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b56e86e-18f8-4523-89d5-8d71e95f611a None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:52:47.162013 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b56e86e-18f8-4523-89d5-8d71e95f611a None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:52:47.162303 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b56e86e-18f8-4523-89d5-8d71e95f611a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:52:47.183317 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-3b56e86e-18f8-4523-89d5-8d71e95f611a None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:52:47.184529 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-3b56e86e-18f8-4523-89d5-8d71e95f611a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:52:47.193269 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1356/2696] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:52:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:52:47.198035 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae8ba87-cd87-4d5e-ad68-1e765d34c5db None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:52:47.207362 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae8ba87-cd87-4d5e-ad68-1e765d34c5db None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:52:47.212021 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d7c16f-c882-4077-bbe8-cd99a2ff9a66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:52:47.212674 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d7c16f-c882-4077-bbe8-cd99a2ff9a66 admin admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:52:47.212956 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d7c16f-c882-4077-bbe8-cd99a2ff9a66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:52:47.213236 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d7c16f-c882-4077-bbe8-cd99a2ff9a66 admin admin] PATH_INFO: `/v3/domains` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:52:47.213887 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d7c16f-c882-4077-bbe8-cd99a2ff9a66 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:52:47.214193 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d7c16f-c882-4077-bbe8-cd99a2ff9a66 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:52:47.215792 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d7c16f-c882-4077-bbe8-cd99a2ff9a66 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:52:47.227411 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1341/2697] 10.222.0.45 () {58 vars in 1153 bytes} [Wed Aug 24 17:52:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:52:47.231911 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b56e86e-18f8-4523-89d5-8d71e95f611a None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:52:47.241666 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b56e86e-18f8-4523-89d5-8d71e95f611a None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:52:47.246239 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bc2682-d04c-46ac-9865-802c0e878a53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:52:47.247151 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bc2682-d04c-46ac-9865-802c0e878a53 admin admin] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:52:47.247293 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bc2682-d04c-46ac-9865-802c0e878a53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:52:47.247461 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bc2682-d04c-46ac-9865-802c0e878a53 admin admin] PATH_INFO: `/v3/projects` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:52:47.248065 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bc2682-d04c-46ac-9865-802c0e878a53 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:52:47.248184 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d7c16f-c882-4077-bbe8-cd99a2ff9a66 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:52:47.249855 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-72d7c16f-c882-4077-bbe8-cd99a2ff9a66 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:52:47.249971 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bc2682-d04c-46ac-9865-802c0e878a53 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:52:47.278132 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1357/2698] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 17:52:47 2022] POST /identity/v3/projects => generated 385 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:52:47.282473 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bc2682-d04c-46ac-9865-802c0e878a53 admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:52:47.314372 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d7c16f-c882-4077-bbe8-cd99a2ff9a66 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:52:47.348098 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bc2682-d04c-46ac-9865-802c0e878a53 admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:52:47.352686 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cabda3e-9c0d-41c9-8da1-db7523449212 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:52:47.353284 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cabda3e-9c0d-41c9-8da1-db7523449212 admin admin] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:52:47.353558 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cabda3e-9c0d-41c9-8da1-db7523449212 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:52:47.353827 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cabda3e-9c0d-41c9-8da1-db7523449212 admin admin] PATH_INFO: `/v3/users` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:52:47.354445 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cabda3e-9c0d-41c9-8da1-db7523449212 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:52:47.356341 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cabda3e-9c0d-41c9-8da1-db7523449212 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:52:47.364907 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9e9b7d-d418-43b9-834f-cd081668b7a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43ad98a250a144099ccd18378aedd4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:52:47.365526 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9e9b7d-d418-43b9-834f-cd081668b7a1 service neutron] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:52:47.365807 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9e9b7d-d418-43b9-834f-cd081668b7a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:52:47.366106 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9e9b7d-d418-43b9-834f-cd081668b7a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:52:47.368380 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-2e9e9b7d-d418-43b9-834f-cd081668b7a1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:52:47.380437 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1358/2699] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 17:52:47 2022] POST /identity/v3/users => generated 293 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:52:47.380636 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9e9b7d-d418-43b9-834f-cd081668b7a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:52:47.380911 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9e9b7d-d418-43b9-834f-cd081668b7a1 service neutron] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:52:47.381531 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cabda3e-9c0d-41c9-8da1-db7523449212 admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:52:47.389259 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1342/2700] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:52:47 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 17:52:47.390940 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cabda3e-9c0d-41c9-8da1-db7523449212 admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:52:47.395651 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a76613f-f12d-4e45-b2db-4e77bf0dc0c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:52:47.396333 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a76613f-f12d-4e45-b2db-4e77bf0dc0c8 admin admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:52:47.396677 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a76613f-f12d-4e45-b2db-4e77bf0dc0c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:52:47.396956 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a76613f-f12d-4e45-b2db-4e77bf0dc0c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:52:47.397629 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a76613f-f12d-4e45-b2db-4e77bf0dc0c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:52:47.399277 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a76613f-f12d-4e45-b2db-4e77bf0dc0c8 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:52:47.410188 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1359/2701] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:52:47 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:52:47.413990 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9e9b7d-d418-43b9-834f-cd081668b7a1 service neutron] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:52:47.425876 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9e9b7d-d418-43b9-834f-cd081668b7a1 service neutron] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:52:47.430397 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d267e00b-1847-435b-ac99-ee012cc0c01e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:52:47.430981 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d267e00b-1847-435b-ac99-ee012cc0c01e admin admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:52:47.431261 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d267e00b-1847-435b-ac99-ee012cc0c01e admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:52:47.431530 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d267e00b-1847-435b-ac99-ee012cc0c01e admin admin] PATH_INFO: `/v3/roles` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:52:47.432163 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d267e00b-1847-435b-ac99-ee012cc0c01e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:52:47.433853 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d267e00b-1847-435b-ac99-ee012cc0c01e admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:52:47.443549 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1343/2702] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:52:47 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:52:47.447668 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a76613f-f12d-4e45-b2db-4e77bf0dc0c8 admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:52:47.457576 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a76613f-f12d-4e45-b2db-4e77bf0dc0c8 admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:52:47.462153 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad94768d-9f1b-4723-a1e3-3e0dec5ee33e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:52:47.462833 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad94768d-9f1b-4723-a1e3-3e0dec5ee33e admin admin] REQUEST_METHOD: `PUT` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:52:47.463111 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad94768d-9f1b-4723-a1e3-3e0dec5ee33e admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:52:47.463385 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad94768d-9f1b-4723-a1e3-3e0dec5ee33e admin admin] PATH_INFO: `/v3/projects/2de47efc59f04643ab02651d40b77067/users/127dbee9996d4c9e8f8603edaa579e36/roles/02476a5b82eb4b188c18a5d706a3ef68` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:52:47.478591 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad94768d-9f1b-4723-a1e3-3e0dec5ee33e admin admin] RBAC: Authorizing `identity:create_grant(project_id=2de47efc59f04643ab02651d40b77067, user_id=127dbee9996d4c9e8f8603edaa579e36, role_id=02476a5b82eb4b188c18a5d706a3ef68)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:52:47.481265 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad94768d-9f1b-4723-a1e3-3e0dec5ee33e admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:52:47.494810 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1360/2703] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 17:52:47 2022] PUT /identity/v3/projects/2de47efc59f04643ab02651d40b77067/users/127dbee9996d4c9e8f8603edaa579e36/roles/02476a5b82eb4b188c18a5d706a3ef68 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:52:47.500132 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11e939f-d708-4f9a-88e6-b541455f1596 None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:52:47.500420 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11e939f-d708-4f9a-88e6-b541455f1596 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:52:47.500722 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11e939f-d708-4f9a-88e6-b541455f1596 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:52:47.519695 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-d11e939f-d708-4f9a-88e6-b541455f1596 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:52:47.520925 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-d11e939f-d708-4f9a-88e6-b541455f1596 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:52:47.559455 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:52:47.581126 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1344/2704] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:52:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:52:47.590938 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad94768d-9f1b-4723-a1e3-3e0dec5ee33e admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:52:47.592640 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-ad94768d-9f1b-4723-a1e3-3e0dec5ee33e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:52:47.647874 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:DELETE=5,keystone:SELECT=89,keystone:INSERT=6 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:52:47.652139 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad94768d-9f1b-4723-a1e3-3e0dec5ee33e admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:52:47.657438 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ccfcd9-97d4-49f6-a271-92f62f8c9281 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43ad98a250a144099ccd18378aedd4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:52:47.658638 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ccfcd9-97d4-49f6-a271-92f62f8c9281 service neutron] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:52:47.659103 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ccfcd9-97d4-49f6-a271-92f62f8c9281 service neutron] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:52:47.659542 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ccfcd9-97d4-49f6-a271-92f62f8c9281 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:52:47.670189 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ccfcd9-97d4-49f6-a271-92f62f8c9281 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:52:47.671598 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ccfcd9-97d4-49f6-a271-92f62f8c9281 service neutron] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:52:47.681299 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1361/2705] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:52:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:52:49.745441 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11e939f-d708-4f9a-88e6-b541455f1596 None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:52:49.747014 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-d11e939f-d708-4f9a-88e6-b541455f1596 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:52:49.874539 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11e939f-d708-4f9a-88e6-b541455f1596 None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:52:49.885048 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d423f1-0cd2-4f18-8566-e89260c71804 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37def4721b554b9ebb7704a19642d15b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:52:49.885998 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d423f1-0cd2-4f18-8566-e89260c71804 service placement] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:52:49.886439 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d423f1-0cd2-4f18-8566-e89260c71804 service placement] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:52:49.886935 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d423f1-0cd2-4f18-8566-e89260c71804 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:52:49.888992 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-29d423f1-0cd2-4f18-8566-e89260c71804 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:52:49.902125 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d423f1-0cd2-4f18-8566-e89260c71804 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:52:49.903903 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d423f1-0cd2-4f18-8566-e89260c71804 service placement] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:52:49.915853 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1345/2706] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:52:49 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:52:51.671601 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f768670-6028-4b46-9c38-1abcab140aa9 None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:52:51.672148 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f768670-6028-4b46-9c38-1abcab140aa9 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:52:51.672370 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f768670-6028-4b46-9c38-1abcab140aa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:52:51.700444 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-9f768670-6028-4b46-9c38-1abcab140aa9 None None] MFA Rules not processed for user `127dbee9996d4c9e8f8603edaa579e36`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:52:51.701769 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-9f768670-6028-4b46-9c38-1abcab140aa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:52:51.769456 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1362/2707] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:52:51 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:52:51.778024 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d423f1-0cd2-4f18-8566-e89260c71804 service placement] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:52:51.801065 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d423f1-0cd2-4f18-8566-e89260c71804 service placement] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:52:51.805747 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f8add6-e203-474b-98b6-29728b0365d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:52:51.806359 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f8add6-e203-474b-98b6-29728b0365d6 admin admin] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:52:51.806699 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f8add6-e203-474b-98b6-29728b0365d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:52:51.806978 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f8add6-e203-474b-98b6-29728b0365d6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:52:51.807727 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f8add6-e203-474b-98b6-29728b0365d6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:52:51.809605 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f8add6-e203-474b-98b6-29728b0365d6 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:52:51.847365 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1346/2708] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 17:52:51 2022] POST /identity/v3/projects => generated 385 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:52:51.851749 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f768670-6028-4b46-9c38-1abcab140aa9 None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:52:51.863117 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f768670-6028-4b46-9c38-1abcab140aa9 None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:52:51.927858 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134722c9-b97d-4f39-a810-3a51384cf777 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:52:51.928329 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134722c9-b97d-4f39-a810-3a51384cf777 admin admin] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:52:51.928647 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134722c9-b97d-4f39-a810-3a51384cf777 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:52:51.928946 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134722c9-b97d-4f39-a810-3a51384cf777 admin admin] PATH_INFO: `/v3/users` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:52:51.929609 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134722c9-b97d-4f39-a810-3a51384cf777 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:52:51.931659 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134722c9-b97d-4f39-a810-3a51384cf777 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:52:51.957398 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1363/2709] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 17:52:51 2022] POST /identity/v3/users => generated 299 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:52:51.961538 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f8add6-e203-474b-98b6-29728b0365d6 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:52:51.976014 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f8add6-e203-474b-98b6-29728b0365d6 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:52:51.980840 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c387d335-da2b-4589-b5b1-e261448420ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:52:51.981468 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c387d335-da2b-4589-b5b1-e261448420ca admin admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:52:51.981756 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c387d335-da2b-4589-b5b1-e261448420ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:52:51.982067 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c387d335-da2b-4589-b5b1-e261448420ca admin admin] PATH_INFO: `/v3/roles` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:52:51.982727 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c387d335-da2b-4589-b5b1-e261448420ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:52:51.984356 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c387d335-da2b-4589-b5b1-e261448420ca admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:52:51.996913 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1347/2710] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:52:51 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:52:52.001696 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134722c9-b97d-4f39-a810-3a51384cf777 admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:52:52.013174 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134722c9-b97d-4f39-a810-3a51384cf777 admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:52:52.019296 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b2e7b9-d6f3-4b99-8be0-b4b6622355cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:52:52.020093 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b2e7b9-d6f3-4b99-8be0-b4b6622355cf admin admin] REQUEST_METHOD: `PUT` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:52:52.020316 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b2e7b9-d6f3-4b99-8be0-b4b6622355cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:52:52.020620 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b2e7b9-d6f3-4b99-8be0-b4b6622355cf admin admin] PATH_INFO: `/v3/projects/f0b13a0288ad488582b72921a21c51a1/users/e5da8016c3624ee5aa9766f0dbdca61e/roles/e411ad9eb96e4e8d85b59b5b700ada7b` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:52:52.037939 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b2e7b9-d6f3-4b99-8be0-b4b6622355cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=f0b13a0288ad488582b72921a21c51a1, user_id=e5da8016c3624ee5aa9766f0dbdca61e, role_id=e411ad9eb96e4e8d85b59b5b700ada7b)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:52:52.040660 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b2e7b9-d6f3-4b99-8be0-b4b6622355cf admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:52:52.055229 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1364/2711] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 17:52:51 2022] PUT /identity/v3/projects/f0b13a0288ad488582b72921a21c51a1/users/e5da8016c3624ee5aa9766f0dbdca61e/roles/e411ad9eb96e4e8d85b59b5b700ada7b => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:52:52.059410 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c387d335-da2b-4589-b5b1-e261448420ca admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:52:52.070552 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c387d335-da2b-4589-b5b1-e261448420ca admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:52:52.130572 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e0f90e-a0ba-42f4-8915-8940ba261b5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:52:52.131184 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e0f90e-a0ba-42f4-8915-8940ba261b5e admin admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:52:52.131489 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e0f90e-a0ba-42f4-8915-8940ba261b5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:52:52.131760 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e0f90e-a0ba-42f4-8915-8940ba261b5e admin admin] PATH_INFO: `/v3/roles` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:52:52.132401 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e0f90e-a0ba-42f4-8915-8940ba261b5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:52:52.134034 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e0f90e-a0ba-42f4-8915-8940ba261b5e admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:52:52.144738 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1348/2712] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:52:52 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:52:52.149311 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b2e7b9-d6f3-4b99-8be0-b4b6622355cf admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:52:52.160129 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b2e7b9-d6f3-4b99-8be0-b4b6622355cf admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:52:52.164956 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4abedd9-f025-44f3-9dda-e269251c7bf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:52:52.165717 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4abedd9-f025-44f3-9dda-e269251c7bf7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:52:52.166063 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4abedd9-f025-44f3-9dda-e269251c7bf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:52:52.166405 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4abedd9-f025-44f3-9dda-e269251c7bf7 admin admin] PATH_INFO: `/v3/projects/f0b13a0288ad488582b72921a21c51a1/users/e5da8016c3624ee5aa9766f0dbdca61e/roles/e411ad9eb96e4e8d85b59b5b700ada7b` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:52:52.168418 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4abedd9-f025-44f3-9dda-e269251c7bf7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f0b13a0288ad488582b72921a21c51a1, user_id=e5da8016c3624ee5aa9766f0dbdca61e, role_id=e411ad9eb96e4e8d85b59b5b700ada7b)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:52:52.171306 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4abedd9-f025-44f3-9dda-e269251c7bf7 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:52:52.184188 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1365/2713] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 17:52:52 2022] PUT /identity/v3/projects/f0b13a0288ad488582b72921a21c51a1/users/e5da8016c3624ee5aa9766f0dbdca61e/roles/e411ad9eb96e4e8d85b59b5b700ada7b => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:52:58.562264 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b792a87-a820-4a78-820d-dfd46f9c24ad None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:52:58.562818 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b792a87-a820-4a78-820d-dfd46f9c24ad None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:52:58.562818 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b792a87-a820-4a78-820d-dfd46f9c24ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:52:58.594164 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-4b792a87-a820-4a78-820d-dfd46f9c24ad None None] MFA Rules not processed for user `e5da8016c3624ee5aa9766f0dbdca61e`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:52:58.595546 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-4b792a87-a820-4a78-820d-dfd46f9c24ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:52:58.624997 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:52:58.670015 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1349/2714] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:52:58 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:52:58.682487 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4abedd9-f025-44f3-9dda-e269251c7bf7 admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:52:58.684344 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-d4abedd9-f025-44f3-9dda-e269251c7bf7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:52:58.727434 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:52:58.771754 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4abedd9-f025-44f3-9dda-e269251c7bf7 admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:52:58.775761 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587e2a4a-7fb2-4178-9ef4-7b44ebf07985 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df5fee57539447428dd1c72eab3cfd9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:52:58.776368 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587e2a4a-7fb2-4178-9ef4-7b44ebf07985 service glance] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:52:58.776710 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587e2a4a-7fb2-4178-9ef4-7b44ebf07985 service glance] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:52:58.776999 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587e2a4a-7fb2-4178-9ef4-7b44ebf07985 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:52:58.834388 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587e2a4a-7fb2-4178-9ef4-7b44ebf07985 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:52:58.835507 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587e2a4a-7fb2-4178-9ef4-7b44ebf07985 service glance] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:52:58.845412 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1366/2715] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:52:58 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 17:52:58.967875 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7613b96b-4478-43f1-93ed-e953678b5ffb None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:52:58.968159 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7613b96b-4478-43f1-93ed-e953678b5ffb None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:52:58.968458 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7613b96b-4478-43f1-93ed-e953678b5ffb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:52:59.022331 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-7613b96b-4478-43f1-93ed-e953678b5ffb None None] MFA Rules not processed for user `ea143d042ee94ae7a4544d90d18db7db`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:52:59.023539 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-7613b96b-4478-43f1-93ed-e953678b5ffb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:52:59.094405 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1350/2716] 10.222.0.45 () {58 vars in 1009 bytes} [Wed Aug 24 17:52:58 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:52:59.143532 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587e2a4a-7fb2-4178-9ef4-7b44ebf07985 service glance] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:52:59.145116 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-587e2a4a-7fb2-4178-9ef4-7b44ebf07985 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:52:59.235135 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587e2a4a-7fb2-4178-9ef4-7b44ebf07985 service glance] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:52:59.240179 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e911ac4c-756b-4d50-8207-1c982ea49645 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90f0772b64c84faaa689e7272fa97898', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:52:59.240818 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e911ac4c-756b-4d50-8207-1c982ea49645 service swift] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:52:59.241100 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e911ac4c-756b-4d50-8207-1c982ea49645 service swift] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:52:59.241373 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e911ac4c-756b-4d50-8207-1c982ea49645 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:52:59.249460 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e911ac4c-756b-4d50-8207-1c982ea49645 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:52:59.251567 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e911ac4c-756b-4d50-8207-1c982ea49645 service swift] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:52:59.259759 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1367/2717] 10.222.0.45 () {60 vars in 1384 bytes} [Wed Aug 24 17:52:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 17:53:04.071843 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7613b96b-4478-43f1-93ed-e953678b5ffb None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:53:04.073350 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-7613b96b-4478-43f1-93ed-e953678b5ffb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:53:04.187678 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7613b96b-4478-43f1-93ed-e953678b5ffb None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:53:04.194075 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f942699-5914-45a1-abd5-1d428d6d0406 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '053b7ce6d6f649ed91a26e434dab805e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:53:04.194841 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f942699-5914-45a1-abd5-1d428d6d0406 service nova] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:53:04.195198 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f942699-5914-45a1-abd5-1d428d6d0406 service nova] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:53:04.195548 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f942699-5914-45a1-abd5-1d428d6d0406 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:53:04.197587 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-0f942699-5914-45a1-abd5-1d428d6d0406 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:53:04.212607 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f942699-5914-45a1-abd5-1d428d6d0406 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:53:04.214331 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f942699-5914-45a1-abd5-1d428d6d0406 service nova] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:53:04.230066 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1351/2718] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:53:04 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:53:07.899178 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3315ac-83c6-473e-9ef2-48eba9efd36f None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:53:07.899769 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3315ac-83c6-473e-9ef2-48eba9efd36f None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:53:07.899769 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3315ac-83c6-473e-9ef2-48eba9efd36f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:53:07.918583 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-9f3315ac-83c6-473e-9ef2-48eba9efd36f None None] MFA Rules not processed for user `127dbee9996d4c9e8f8603edaa579e36`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:53:07.919864 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-9f3315ac-83c6-473e-9ef2-48eba9efd36f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:53:07.927546 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1368/2719] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:53:07 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:53:08.175607 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f942699-5914-45a1-abd5-1d428d6d0406 service nova] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:53:08.194947 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:53:08.231070 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f942699-5914-45a1-abd5-1d428d6d0406 service nova] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:53:08.235114 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9886bab-da97-42cd-a81c-28fda14366d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43ad98a250a144099ccd18378aedd4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:53:08.235725 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9886bab-da97-42cd-a81c-28fda14366d8 service neutron] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:53:08.236005 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9886bab-da97-42cd-a81c-28fda14366d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:53:08.236282 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9886bab-da97-42cd-a81c-28fda14366d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:53:08.238049 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-d9886bab-da97-42cd-a81c-28fda14366d8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:53:08.247559 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9886bab-da97-42cd-a81c-28fda14366d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:53:08.249642 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9886bab-da97-42cd-a81c-28fda14366d8 service neutron] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:53:08.257798 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1352/2720] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:53:08 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:53:09.852574 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14811ae4-608b-441e-8c79-2e6b72a64ce9 None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:53:09.853070 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14811ae4-608b-441e-8c79-2e6b72a64ce9 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:53:09.853264 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14811ae4-608b-441e-8c79-2e6b72a64ce9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:53:09.873921 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-14811ae4-608b-441e-8c79-2e6b72a64ce9 None None] MFA Rules not processed for user `ea143d042ee94ae7a4544d90d18db7db`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:53:09.875141 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-14811ae4-608b-441e-8c79-2e6b72a64ce9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:53:09.883237 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1369/2721] 10.222.0.45 () {58 vars in 1009 bytes} [Wed Aug 24 17:53:09 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:53:09.895366 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9886bab-da97-42cd-a81c-28fda14366d8 service neutron] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:53:09.908454 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9886bab-da97-42cd-a81c-28fda14366d8 service neutron] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:53:09.914618 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac20a185-4960-4fdc-b245-bbd9261abaf1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90f0772b64c84faaa689e7272fa97898', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:53:09.915437 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac20a185-4960-4fdc-b245-bbd9261abaf1 service swift] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:53:09.915790 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac20a185-4960-4fdc-b245-bbd9261abaf1 service swift] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:53:09.916139 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac20a185-4960-4fdc-b245-bbd9261abaf1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:53:09.925151 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac20a185-4960-4fdc-b245-bbd9261abaf1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:53:09.926903 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac20a185-4960-4fdc-b245-bbd9261abaf1 service swift] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:53:09.935265 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1353/2722] 10.222.0.45 () {60 vars in 1384 bytes} [Wed Aug 24 17:53:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 17:53:19.867530 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=89 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:53:19.931738 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=24 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:53:35.915624 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14811ae4-608b-441e-8c79-2e6b72a64ce9 None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:53:35.979289 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14811ae4-608b-441e-8c79-2e6b72a64ce9 None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:53:35.984763 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88098d30-67c7-4366-9f1b-bd8fba6d04d8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37def4721b554b9ebb7704a19642d15b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:53:35.985389 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88098d30-67c7-4366-9f1b-bd8fba6d04d8 service placement] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:53:35.985687 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88098d30-67c7-4366-9f1b-bd8fba6d04d8 service placement] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:53:35.985949 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88098d30-67c7-4366-9f1b-bd8fba6d04d8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:53:35.987751 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-88098d30-67c7-4366-9f1b-bd8fba6d04d8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:53:35.998970 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88098d30-67c7-4366-9f1b-bd8fba6d04d8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:53:36.000524 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88098d30-67c7-4366-9f1b-bd8fba6d04d8 service placement] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:53:36.010012 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1370/2723] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:53:35 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:53:40.272384 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac20a185-4960-4fdc-b245-bbd9261abaf1 service swift] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:53:40.281854 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac20a185-4960-4fdc-b245-bbd9261abaf1 service swift] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:53:40.285634 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f691b8-7ee9-4986-be1d-95eceffce839 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43ad98a250a144099ccd18378aedd4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:53:40.286232 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f691b8-7ee9-4986-be1d-95eceffce839 service neutron] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:53:40.286508 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f691b8-7ee9-4986-be1d-95eceffce839 service neutron] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:53:40.286779 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f691b8-7ee9-4986-be1d-95eceffce839 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:53:40.296114 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f691b8-7ee9-4986-be1d-95eceffce839 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:53:40.297645 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f691b8-7ee9-4986-be1d-95eceffce839 service neutron] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:53:40.306886 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1354/2724] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:53:40 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 17:53:46.005913 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=32 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:53:50.302818 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=6 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:54:53.044035 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88098d30-67c7-4366-9f1b-bd8fba6d04d8 service placement] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:54:53.053136 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88098d30-67c7-4366-9f1b-bd8fba6d04d8 service placement] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:54:53.057222 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe8a1e5-312f-4dc6-9887-e21795b83014 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43ad98a250a144099ccd18378aedd4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:54:53.057807 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe8a1e5-312f-4dc6-9887-e21795b83014 service neutron] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:54:53.058081 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe8a1e5-312f-4dc6-9887-e21795b83014 service neutron] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:54:53.058351 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe8a1e5-312f-4dc6-9887-e21795b83014 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:54:53.060021 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-abe8a1e5-312f-4dc6-9887-e21795b83014 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:54:53.121611 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe8a1e5-312f-4dc6-9887-e21795b83014 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:54:53.123049 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe8a1e5-312f-4dc6-9887-e21795b83014 service neutron] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:54:53.132584 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1371/2725] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:54:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:54:56.755161 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f691b8-7ee9-4986-be1d-95eceffce839 service neutron] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:54:56.756712 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-d1f691b8-7ee9-4986-be1d-95eceffce839 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:54:56.770946 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f691b8-7ee9-4986-be1d-95eceffce839 service neutron] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:54:56.776154 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c262b834-9f17-4746-897e-fb8b0c4b15a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:54:56.776860 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c262b834-9f17-4746-897e-fb8b0c4b15a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:54:56.777142 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c262b834-9f17-4746-897e-fb8b0c4b15a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:54:56.777422 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c262b834-9f17-4746-897e-fb8b0c4b15a7 admin admin] PATH_INFO: `/v3/users/58652b0d60ec4134be18dafeb7ad5cb0` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:54:56.778614 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c262b834-9f17-4746-897e-fb8b0c4b15a7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=58652b0d60ec4134be18dafeb7ad5cb0)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:54:56.780544 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c262b834-9f17-4746-897e-fb8b0c4b15a7 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:54:56.830152 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-c262b834-9f17-4746-897e-fb8b0c4b15a7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '58652b0d60ec4134be18dafeb7ad5cb0'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:54:56.830783 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-c262b834-9f17-4746-897e-fb8b0c4b15a7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '58652b0d60ec4134be18dafeb7ad5cb0'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:54:56.838709 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-c262b834-9f17-4746-897e-fb8b0c4b15a7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '58652b0d60ec4134be18dafeb7ad5cb0'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:54:56.839332 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-c262b834-9f17-4746-897e-fb8b0c4b15a7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '58652b0d60ec4134be18dafeb7ad5cb0'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:54:56.846760 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-c262b834-9f17-4746-897e-fb8b0c4b15a7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '58652b0d60ec4134be18dafeb7ad5cb0'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:54:56.864261 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1355/2726] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 17:54:56 2022] DELETE /identity/v3/users/58652b0d60ec4134be18dafeb7ad5cb0 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:54:57.042168 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe8a1e5-312f-4dc6-9887-e21795b83014 service neutron] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:54:57.053497 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe8a1e5-312f-4dc6-9887-e21795b83014 service neutron] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:54:57.110421 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a49cc8-ebe4-454d-85e4-f62ff4fe6e3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:54:57.111078 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a49cc8-ebe4-454d-85e4-f62ff4fe6e3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:54:57.111359 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a49cc8-ebe4-454d-85e4-f62ff4fe6e3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:54:57.111637 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a49cc8-ebe4-454d-85e4-f62ff4fe6e3d admin admin] PATH_INFO: `/v3/projects/75c39bcd24354fd5b79179c48abc07f1` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:54:57.112797 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a49cc8-ebe4-454d-85e4-f62ff4fe6e3d admin admin] RBAC: Authorizing `identity:delete_project(project_id=75c39bcd24354fd5b79179c48abc07f1)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:54:57.114698 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a49cc8-ebe4-454d-85e4-f62ff4fe6e3d admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:54:57.159883 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-e6a49cc8-ebe4-454d-85e4-f62ff4fe6e3d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '75c39bcd24354fd5b79179c48abc07f1', 'request_id': 'req-e6a49cc8-ebe4-454d-85e4-f62ff4fe6e3d'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:54:57.167476 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-e6a49cc8-ebe4-454d-85e4-f62ff4fe6e3d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '75c39bcd24354fd5b79179c48abc07f1', 'request_id': 'req-e6a49cc8-ebe4-454d-85e4-f62ff4fe6e3d'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:54:57.173545 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-e6a49cc8-ebe4-454d-85e4-f62ff4fe6e3d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '75c39bcd24354fd5b79179c48abc07f1', 'request_id': 'req-e6a49cc8-ebe4-454d-85e4-f62ff4fe6e3d'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:54:57.183537 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-e6a49cc8-ebe4-454d-85e4-f62ff4fe6e3d admin admin] The token cache is being invalidate because project 75c39bcd24354fd5b79179c48abc07f1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81758) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:54:57.183824 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-e6a49cc8-ebe4-454d-85e4-f62ff4fe6e3d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:54:57.184441 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-e6a49cc8-ebe4-454d-85e4-f62ff4fe6e3d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:54:57.186198 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1372/2727] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 17:54:57 2022] DELETE /identity/v3/projects/75c39bcd24354fd5b79179c48abc07f1 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:54:57.190891 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9595009c-407e-465b-b417-9ec3ae2a443b None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:54:57.191178 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9595009c-407e-465b-b417-9ec3ae2a443b None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:54:57.191455 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9595009c-407e-465b-b417-9ec3ae2a443b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:54:57.211741 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-9595009c-407e-465b-b417-9ec3ae2a443b None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:54:57.212990 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-9595009c-407e-465b-b417-9ec3ae2a443b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:54:57.276729 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1356/2728] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:54:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:54:57.285032 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fb0dac-403c-4eed-8b7a-0d578396e942 None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:54:57.285320 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fb0dac-403c-4eed-8b7a-0d578396e942 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:54:57.285596 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fb0dac-403c-4eed-8b7a-0d578396e942 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:54:57.306005 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-90fb0dac-403c-4eed-8b7a-0d578396e942 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:54:57.307364 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-90fb0dac-403c-4eed-8b7a-0d578396e942 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:54:57.315492 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1373/2729] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:54:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:54:57.319937 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9595009c-407e-465b-b417-9ec3ae2a443b None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:54:57.330102 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9595009c-407e-465b-b417-9ec3ae2a443b None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:54:57.334752 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2544fd7-26ef-4746-8710-c14f4128e11e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:54:57.335347 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2544fd7-26ef-4746-8710-c14f4128e11e admin admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:54:57.335627 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2544fd7-26ef-4746-8710-c14f4128e11e admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:54:57.335898 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2544fd7-26ef-4746-8710-c14f4128e11e admin admin] PATH_INFO: `/v3/domains` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:54:57.337148 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2544fd7-26ef-4746-8710-c14f4128e11e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:54:57.337453 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2544fd7-26ef-4746-8710-c14f4128e11e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:54:57.339041 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2544fd7-26ef-4746-8710-c14f4128e11e admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:54:57.350334 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1357/2730] 10.222.0.45 () {58 vars in 1153 bytes} [Wed Aug 24 17:54:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:54:57.354620 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fb0dac-403c-4eed-8b7a-0d578396e942 None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:54:57.363703 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fb0dac-403c-4eed-8b7a-0d578396e942 None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:54:57.368245 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac496962-3ec8-4e78-a72e-1c13588052df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:54:57.368867 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac496962-3ec8-4e78-a72e-1c13588052df admin admin] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:54:57.369143 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac496962-3ec8-4e78-a72e-1c13588052df admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:54:57.369417 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac496962-3ec8-4e78-a72e-1c13588052df admin admin] PATH_INFO: `/v3/projects` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:54:57.370037 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac496962-3ec8-4e78-a72e-1c13588052df admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:54:57.371899 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac496962-3ec8-4e78-a72e-1c13588052df admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:54:57.387565 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:54:57.402746 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1374/2731] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 17:54:57 2022] POST /identity/v3/projects => generated 403 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:54:57.406991 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2544fd7-26ef-4746-8710-c14f4128e11e admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:54:57.417029 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2544fd7-26ef-4746-8710-c14f4128e11e admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:54:57.471268 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584a2666-3da7-4cbe-9023-e8475550e163 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:54:57.471868 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584a2666-3da7-4cbe-9023-e8475550e163 admin admin] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:54:57.472145 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584a2666-3da7-4cbe-9023-e8475550e163 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:54:57.472423 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584a2666-3da7-4cbe-9023-e8475550e163 admin admin] PATH_INFO: `/v3/users` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:54:57.473072 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584a2666-3da7-4cbe-9023-e8475550e163 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:54:57.474984 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584a2666-3da7-4cbe-9023-e8475550e163 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:54:57.484027 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:54:57.497872 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1358/2732] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 17:54:57 2022] POST /identity/v3/users => generated 302 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:54:57.502241 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac496962-3ec8-4e78-a72e-1c13588052df admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:54:57.513396 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac496962-3ec8-4e78-a72e-1c13588052df admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:54:57.518329 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b3e429-3768-47c7-8b77-81734783b7d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:54:57.518999 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b3e429-3768-47c7-8b77-81734783b7d5 admin admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:54:57.519344 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b3e429-3768-47c7-8b77-81734783b7d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:54:57.519683 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b3e429-3768-47c7-8b77-81734783b7d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:54:57.520399 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b3e429-3768-47c7-8b77-81734783b7d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:54:57.522028 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b3e429-3768-47c7-8b77-81734783b7d5 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:54:57.533216 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1375/2733] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:54:57 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 17:54:57.537555 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584a2666-3da7-4cbe-9023-e8475550e163 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:54:57.548034 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584a2666-3da7-4cbe-9023-e8475550e163 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:54:57.552781 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2347e9aa-8fcd-4f72-be3c-38c576b36668 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:54:57.553391 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2347e9aa-8fcd-4f72-be3c-38c576b36668 admin admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:54:57.553673 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2347e9aa-8fcd-4f72-be3c-38c576b36668 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:54:57.553952 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2347e9aa-8fcd-4f72-be3c-38c576b36668 admin admin] PATH_INFO: `/v3/roles` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:54:57.555242 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2347e9aa-8fcd-4f72-be3c-38c576b36668 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:54:57.556872 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2347e9aa-8fcd-4f72-be3c-38c576b36668 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:54:57.571527 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1359/2734] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:54:57 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 17:54:57.572995 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b3e429-3768-47c7-8b77-81734783b7d5 admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:54:57.586740 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b3e429-3768-47c7-8b77-81734783b7d5 admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:54:57.591878 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d364945-3ba8-4b6d-b300-fa5dadb5e7aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:54:57.592569 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d364945-3ba8-4b6d-b300-fa5dadb5e7aa admin admin] REQUEST_METHOD: `PUT` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:54:57.592904 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d364945-3ba8-4b6d-b300-fa5dadb5e7aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:54:57.593183 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d364945-3ba8-4b6d-b300-fa5dadb5e7aa admin admin] PATH_INFO: `/v3/projects/eb0dccad85944c97b0f0b22261fab750/users/c18ff666f7134d4fa16648b31175927d/roles/02476a5b82eb4b188c18a5d706a3ef68` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:54:57.608020 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d364945-3ba8-4b6d-b300-fa5dadb5e7aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb0dccad85944c97b0f0b22261fab750, user_id=c18ff666f7134d4fa16648b31175927d, role_id=02476a5b82eb4b188c18a5d706a3ef68)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:54:57.610715 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d364945-3ba8-4b6d-b300-fa5dadb5e7aa admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:54:57.625187 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1376/2735] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 17:54:57 2022] PUT /identity/v3/projects/eb0dccad85944c97b0f0b22261fab750/users/c18ff666f7134d4fa16648b31175927d/roles/02476a5b82eb4b188c18a5d706a3ef68 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:54:57.630238 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7686cc-7bc3-4476-a4be-2d6ae1eaafba None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:54:57.630737 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7686cc-7bc3-4476-a4be-2d6ae1eaafba None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:54:57.631081 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7686cc-7bc3-4476-a4be-2d6ae1eaafba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:54:57.651353 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-eb7686cc-7bc3-4476-a4be-2d6ae1eaafba None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:54:57.652610 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-eb7686cc-7bc3-4476-a4be-2d6ae1eaafba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:54:57.710092 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1360/2736] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:54:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:54:57.719582 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d364945-3ba8-4b6d-b300-fa5dadb5e7aa admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:54:57.721305 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-6d364945-3ba8-4b6d-b300-fa5dadb5e7aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:54:57.777727 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d364945-3ba8-4b6d-b300-fa5dadb5e7aa admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:54:57.781517 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6457bc-817f-47ac-8766-23edc8b7a4a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43ad98a250a144099ccd18378aedd4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:54:57.782016 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6457bc-817f-47ac-8766-23edc8b7a4a8 service neutron] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:54:57.782290 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6457bc-817f-47ac-8766-23edc8b7a4a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:54:57.782560 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6457bc-817f-47ac-8766-23edc8b7a4a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:54:57.791365 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6457bc-817f-47ac-8766-23edc8b7a4a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:54:57.792838 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6457bc-817f-47ac-8766-23edc8b7a4a8 service neutron] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:54:57.802230 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1377/2737] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:54:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:54:58.400084 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28951463-7cb6-4483-b858-0b466333cac9 None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:54:58.400607 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28951463-7cb6-4483-b858-0b466333cac9 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:54:58.400737 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28951463-7cb6-4483-b858-0b466333cac9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:54:58.429993 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-28951463-7cb6-4483-b858-0b466333cac9 None None] MFA Rules not processed for user `c18ff666f7134d4fa16648b31175927d`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:54:58.431220 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-28951463-7cb6-4483-b858-0b466333cac9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:54:58.501307 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1361/2738] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:54:58 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:54:58.505700 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6457bc-817f-47ac-8766-23edc8b7a4a8 service neutron] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:54:58.515368 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6457bc-817f-47ac-8766-23edc8b7a4a8 service neutron] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:54:58.519958 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1fd00c-e0b5-4783-8cfa-038e27e88d8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:54:58.520490 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1fd00c-e0b5-4783-8cfa-038e27e88d8d admin admin] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:54:58.520809 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1fd00c-e0b5-4783-8cfa-038e27e88d8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:54:58.521085 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1fd00c-e0b5-4783-8cfa-038e27e88d8d admin admin] PATH_INFO: `/v3/projects` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:54:58.521722 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1fd00c-e0b5-4783-8cfa-038e27e88d8d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:54:58.524111 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1fd00c-e0b5-4783-8cfa-038e27e88d8d admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:54:58.555220 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1378/2739] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 17:54:58 2022] POST /identity/v3/projects => generated 403 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:54:58.556418 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28951463-7cb6-4483-b858-0b466333cac9 None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:54:58.567650 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28951463-7cb6-4483-b858-0b466333cac9 None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:54:58.627796 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801b17d6-1bc3-400a-b11f-170b932fcdf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:54:58.628657 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801b17d6-1bc3-400a-b11f-170b932fcdf5 admin admin] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:54:58.629197 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801b17d6-1bc3-400a-b11f-170b932fcdf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:54:58.629490 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801b17d6-1bc3-400a-b11f-170b932fcdf5 admin admin] PATH_INFO: `/v3/users` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:54:58.630143 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801b17d6-1bc3-400a-b11f-170b932fcdf5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:54:58.632095 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801b17d6-1bc3-400a-b11f-170b932fcdf5 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:54:58.650200 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:54:58.657484 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1362/2740] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 17:54:58 2022] POST /identity/v3/users => generated 308 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:54:58.661569 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1fd00c-e0b5-4783-8cfa-038e27e88d8d admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:54:58.673286 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1fd00c-e0b5-4783-8cfa-038e27e88d8d admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:54:58.678358 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f26ed5-e587-43b7-a309-1e21c4daa07f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:54:58.679048 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f26ed5-e587-43b7-a309-1e21c4daa07f admin admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:54:58.679354 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f26ed5-e587-43b7-a309-1e21c4daa07f admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:54:58.679688 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f26ed5-e587-43b7-a309-1e21c4daa07f admin admin] PATH_INFO: `/v3/roles` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:54:58.681005 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f26ed5-e587-43b7-a309-1e21c4daa07f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:54:58.682631 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f26ed5-e587-43b7-a309-1e21c4daa07f admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:54:58.697310 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1379/2741] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:54:58 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:54:58.698689 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801b17d6-1bc3-400a-b11f-170b932fcdf5 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:54:58.715110 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801b17d6-1bc3-400a-b11f-170b932fcdf5 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:54:58.720160 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17d318b-7c19-4fd2-875c-e51337595b95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:54:58.720958 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17d318b-7c19-4fd2-875c-e51337595b95 admin admin] REQUEST_METHOD: `PUT` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:54:58.721304 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17d318b-7c19-4fd2-875c-e51337595b95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:54:58.721645 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17d318b-7c19-4fd2-875c-e51337595b95 admin admin] PATH_INFO: `/v3/projects/9e3198addb7b4b2ab4e2e0f1c2c504b8/users/6804bd2e8e524afd9b4111cfeccacbcd/roles/e411ad9eb96e4e8d85b59b5b700ada7b` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:54:58.737726 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17d318b-7c19-4fd2-875c-e51337595b95 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e3198addb7b4b2ab4e2e0f1c2c504b8, user_id=6804bd2e8e524afd9b4111cfeccacbcd, role_id=e411ad9eb96e4e8d85b59b5b700ada7b)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:54:58.740372 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17d318b-7c19-4fd2-875c-e51337595b95 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:54:58.755264 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1363/2742] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 17:54:58 2022] PUT /identity/v3/projects/9e3198addb7b4b2ab4e2e0f1c2c504b8/users/6804bd2e8e524afd9b4111cfeccacbcd/roles/e411ad9eb96e4e8d85b59b5b700ada7b => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:54:58.759369 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f26ed5-e587-43b7-a309-1e21c4daa07f admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:54:58.769785 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f26ed5-e587-43b7-a309-1e21c4daa07f admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:54:58.827973 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e15d73e-767b-45bc-9456-21248c28d3a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:54:58.828618 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e15d73e-767b-45bc-9456-21248c28d3a7 admin admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:54:58.828902 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e15d73e-767b-45bc-9456-21248c28d3a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:54:58.829177 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e15d73e-767b-45bc-9456-21248c28d3a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:54:58.829818 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e15d73e-767b-45bc-9456-21248c28d3a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:54:58.831404 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e15d73e-767b-45bc-9456-21248c28d3a7 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:54:58.844359 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1380/2743] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:54:58 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:54:58.845357 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17d318b-7c19-4fd2-875c-e51337595b95 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:54:58.856412 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17d318b-7c19-4fd2-875c-e51337595b95 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:54:58.861385 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ec1148-908c-46dd-ae17-4c0f6b227a90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:54:58.862037 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ec1148-908c-46dd-ae17-4c0f6b227a90 admin admin] REQUEST_METHOD: `PUT` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:54:58.862317 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ec1148-908c-46dd-ae17-4c0f6b227a90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:54:58.862592 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ec1148-908c-46dd-ae17-4c0f6b227a90 admin admin] PATH_INFO: `/v3/projects/9e3198addb7b4b2ab4e2e0f1c2c504b8/users/6804bd2e8e524afd9b4111cfeccacbcd/roles/e411ad9eb96e4e8d85b59b5b700ada7b` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:54:58.864604 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ec1148-908c-46dd-ae17-4c0f6b227a90 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e3198addb7b4b2ab4e2e0f1c2c504b8, user_id=6804bd2e8e524afd9b4111cfeccacbcd, role_id=e411ad9eb96e4e8d85b59b5b700ada7b)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:54:58.867289 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ec1148-908c-46dd-ae17-4c0f6b227a90 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:54:58.877550 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1364/2744] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 17:54:58 2022] PUT /identity/v3/projects/9e3198addb7b4b2ab4e2e0f1c2c504b8/users/6804bd2e8e524afd9b4111cfeccacbcd/roles/e411ad9eb96e4e8d85b59b5b700ada7b => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:54:59.710968 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9c5511-0818-4bdd-929f-68afb8225d92 None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:54:59.711531 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9c5511-0818-4bdd-929f-68afb8225d92 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:54:59.711531 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9c5511-0818-4bdd-929f-68afb8225d92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:54:59.739116 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-da9c5511-0818-4bdd-929f-68afb8225d92 None None] MFA Rules not processed for user `6804bd2e8e524afd9b4111cfeccacbcd`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:54:59.740434 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-da9c5511-0818-4bdd-929f-68afb8225d92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:54:59.754022 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:54:59.819180 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1381/2745] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:54:59 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:55:00.818444 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ec1148-908c-46dd-ae17-4c0f6b227a90 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:55:00.882360 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ec1148-908c-46dd-ae17-4c0f6b227a90 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:55:00.887339 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f263810c-4f18-44d9-978d-0a80612e9c5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:55:00.887954 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f263810c-4f18-44d9-978d-0a80612e9c5e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:55:00.888230 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f263810c-4f18-44d9-978d-0a80612e9c5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:55:00.888779 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f263810c-4f18-44d9-978d-0a80612e9c5e admin admin] PATH_INFO: `/v3/users/c18ff666f7134d4fa16648b31175927d` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:55:00.889874 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f263810c-4f18-44d9-978d-0a80612e9c5e admin admin] RBAC: Authorizing `identity:delete_user(user_id=c18ff666f7134d4fa16648b31175927d)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:55:00.891810 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f263810c-4f18-44d9-978d-0a80612e9c5e admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:55:00.936935 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-f263810c-4f18-44d9-978d-0a80612e9c5e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c18ff666f7134d4fa16648b31175927d'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:55:00.937581 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-f263810c-4f18-44d9-978d-0a80612e9c5e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c18ff666f7134d4fa16648b31175927d'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:55:00.944524 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-f263810c-4f18-44d9-978d-0a80612e9c5e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c18ff666f7134d4fa16648b31175927d'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:55:00.945736 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-f263810c-4f18-44d9-978d-0a80612e9c5e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c18ff666f7134d4fa16648b31175927d'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:55:00.952609 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-f263810c-4f18-44d9-978d-0a80612e9c5e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c18ff666f7134d4fa16648b31175927d'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:55:00.970168 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1365/2746] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 17:55:00 2022] DELETE /identity/v3/users/c18ff666f7134d4fa16648b31175927d => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:55:01.143994 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9c5511-0818-4bdd-929f-68afb8225d92 None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:55:01.156903 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9c5511-0818-4bdd-929f-68afb8225d92 None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:55:01.215314 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fd8cc0-128a-4781-b832-ed263db16e55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:55:01.215948 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fd8cc0-128a-4781-b832-ed263db16e55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:55:01.216274 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fd8cc0-128a-4781-b832-ed263db16e55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:55:01.216541 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fd8cc0-128a-4781-b832-ed263db16e55 admin admin] PATH_INFO: `/v3/projects/eb0dccad85944c97b0f0b22261fab750` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:55:01.217671 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fd8cc0-128a-4781-b832-ed263db16e55 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eb0dccad85944c97b0f0b22261fab750)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:55:01.219553 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fd8cc0-128a-4781-b832-ed263db16e55 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:55:01.265401 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-20fd8cc0-128a-4781-b832-ed263db16e55 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eb0dccad85944c97b0f0b22261fab750', 'request_id': 'req-20fd8cc0-128a-4781-b832-ed263db16e55'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:55:01.272449 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-20fd8cc0-128a-4781-b832-ed263db16e55 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eb0dccad85944c97b0f0b22261fab750', 'request_id': 'req-20fd8cc0-128a-4781-b832-ed263db16e55'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:55:01.279146 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-20fd8cc0-128a-4781-b832-ed263db16e55 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eb0dccad85944c97b0f0b22261fab750', 'request_id': 'req-20fd8cc0-128a-4781-b832-ed263db16e55'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:55:01.290530 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-20fd8cc0-128a-4781-b832-ed263db16e55 admin admin] The token cache is being invalidate because project eb0dccad85944c97b0f0b22261fab750 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81758) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:55:01.290851 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-20fd8cc0-128a-4781-b832-ed263db16e55 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:55:01.291517 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-20fd8cc0-128a-4781-b832-ed263db16e55 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:55:01.293179 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1382/2747] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 17:55:01 2022] DELETE /identity/v3/projects/eb0dccad85944c97b0f0b22261fab750 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:55:01.297125 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f263810c-4f18-44d9-978d-0a80612e9c5e admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:55:01.298716 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-f263810c-4f18-44d9-978d-0a80612e9c5e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:55:01.312178 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f263810c-4f18-44d9-978d-0a80612e9c5e admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:55:01.352170 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:55:01.371679 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800df650-bede-4840-8545-30905ad6bab4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:55:01.372303 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800df650-bede-4840-8545-30905ad6bab4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:55:01.372605 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800df650-bede-4840-8545-30905ad6bab4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:55:01.372879 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800df650-bede-4840-8545-30905ad6bab4 admin admin] PATH_INFO: `/v3/users/6804bd2e8e524afd9b4111cfeccacbcd` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:55:01.373984 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800df650-bede-4840-8545-30905ad6bab4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6804bd2e8e524afd9b4111cfeccacbcd)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:55:01.375894 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800df650-bede-4840-8545-30905ad6bab4 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:55:01.419667 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-800df650-bede-4840-8545-30905ad6bab4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6804bd2e8e524afd9b4111cfeccacbcd'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:55:01.420297 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-800df650-bede-4840-8545-30905ad6bab4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6804bd2e8e524afd9b4111cfeccacbcd'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:55:01.427023 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-800df650-bede-4840-8545-30905ad6bab4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6804bd2e8e524afd9b4111cfeccacbcd'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:55:01.427641 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-800df650-bede-4840-8545-30905ad6bab4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6804bd2e8e524afd9b4111cfeccacbcd'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:55:01.435274 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-800df650-bede-4840-8545-30905ad6bab4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6804bd2e8e524afd9b4111cfeccacbcd'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:55:01.453270 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1366/2748] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 17:55:01 2022] DELETE /identity/v3/users/6804bd2e8e524afd9b4111cfeccacbcd => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:55:01.620119 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fd8cc0-128a-4781-b832-ed263db16e55 admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:55:01.621581 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-20fd8cc0-128a-4781-b832-ed263db16e55 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:55:01.638803 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fd8cc0-128a-4781-b832-ed263db16e55 admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:55:01.679943 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:55:01.698587 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07de5f61-77a2-4642-ad81-08238bec34d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:55:01.699213 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07de5f61-77a2-4642-ad81-08238bec34d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:55:01.699484 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07de5f61-77a2-4642-ad81-08238bec34d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:55:01.699753 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07de5f61-77a2-4642-ad81-08238bec34d8 admin admin] PATH_INFO: `/v3/projects/9e3198addb7b4b2ab4e2e0f1c2c504b8` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:55:01.700908 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07de5f61-77a2-4642-ad81-08238bec34d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9e3198addb7b4b2ab4e2e0f1c2c504b8)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:55:01.702800 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07de5f61-77a2-4642-ad81-08238bec34d8 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:55:01.748171 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-07de5f61-77a2-4642-ad81-08238bec34d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9e3198addb7b4b2ab4e2e0f1c2c504b8', 'request_id': 'req-07de5f61-77a2-4642-ad81-08238bec34d8'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:55:01.755060 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-07de5f61-77a2-4642-ad81-08238bec34d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9e3198addb7b4b2ab4e2e0f1c2c504b8', 'request_id': 'req-07de5f61-77a2-4642-ad81-08238bec34d8'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:55:01.761235 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-07de5f61-77a2-4642-ad81-08238bec34d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9e3198addb7b4b2ab4e2e0f1c2c504b8', 'request_id': 'req-07de5f61-77a2-4642-ad81-08238bec34d8'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:55:01.771792 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-07de5f61-77a2-4642-ad81-08238bec34d8 admin admin] The token cache is being invalidate because project 9e3198addb7b4b2ab4e2e0f1c2c504b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81758) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:55:01.772089 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-07de5f61-77a2-4642-ad81-08238bec34d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:55:01.772781 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-07de5f61-77a2-4642-ad81-08238bec34d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:55:01.774265 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1383/2749] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 17:55:01 2022] DELETE /identity/v3/projects/9e3198addb7b4b2ab4e2e0f1c2c504b8 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:55:01.779107 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ee1d5d-712a-49d5-ac14-f52916d1c0ec None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:55:01.779401 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ee1d5d-712a-49d5-ac14-f52916d1c0ec None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:55:01.779696 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ee1d5d-712a-49d5-ac14-f52916d1c0ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:55:01.799724 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-b1ee1d5d-712a-49d5-ac14-f52916d1c0ec None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:55:01.801027 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-b1ee1d5d-712a-49d5-ac14-f52916d1c0ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:55:01.858652 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1367/2750] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:55:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:55:01.866388 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6227779e-795a-4faf-b761-cdc500974a92 None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:55:01.866620 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6227779e-795a-4faf-b761-cdc500974a92 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:55:01.867562 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6227779e-795a-4faf-b761-cdc500974a92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:55:01.886881 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-6227779e-795a-4faf-b761-cdc500974a92 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:55:01.888075 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-6227779e-795a-4faf-b761-cdc500974a92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:55:01.896074 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1384/2751] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:55:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:55:01.900151 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ee1d5d-712a-49d5-ac14-f52916d1c0ec None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:55:01.909766 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ee1d5d-712a-49d5-ac14-f52916d1c0ec None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:55:01.914818 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0711a704-6542-4d05-a6db-a2eb9304ae6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:55:01.915435 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0711a704-6542-4d05-a6db-a2eb9304ae6e admin admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:55:01.915720 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0711a704-6542-4d05-a6db-a2eb9304ae6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:55:01.915989 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0711a704-6542-4d05-a6db-a2eb9304ae6e admin admin] PATH_INFO: `/v3/domains` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:55:01.916682 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0711a704-6542-4d05-a6db-a2eb9304ae6e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:55:01.916986 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0711a704-6542-4d05-a6db-a2eb9304ae6e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:55:01.918578 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0711a704-6542-4d05-a6db-a2eb9304ae6e admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:55:01.931249 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1368/2752] 10.222.0.45 () {58 vars in 1153 bytes} [Wed Aug 24 17:55:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:55:01.935567 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6227779e-795a-4faf-b761-cdc500974a92 None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:55:01.944598 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6227779e-795a-4faf-b761-cdc500974a92 None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:55:01.949232 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897c384a-a930-4045-87bd-e6a76298c22a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:55:01.949813 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897c384a-a930-4045-87bd-e6a76298c22a admin admin] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:55:01.950087 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897c384a-a930-4045-87bd-e6a76298c22a admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:55:01.950357 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897c384a-a930-4045-87bd-e6a76298c22a admin admin] PATH_INFO: `/v3/projects` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:55:01.950975 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897c384a-a930-4045-87bd-e6a76298c22a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:55:01.952910 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897c384a-a930-4045-87bd-e6a76298c22a admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:55:01.979468 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1385/2753] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 17:55:01 2022] POST /identity/v3/projects => generated 373 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:55:01.983546 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0711a704-6542-4d05-a6db-a2eb9304ae6e admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:55:01.994058 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0711a704-6542-4d05-a6db-a2eb9304ae6e admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:55:02.037853 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:55:02.052107 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d54bc49-39d1-499e-9c64-cff68590c793 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:55:02.052743 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d54bc49-39d1-499e-9c64-cff68590c793 admin admin] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:55:02.053024 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d54bc49-39d1-499e-9c64-cff68590c793 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:55:02.053297 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d54bc49-39d1-499e-9c64-cff68590c793 admin admin] PATH_INFO: `/v3/users` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:55:02.053930 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d54bc49-39d1-499e-9c64-cff68590c793 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:55:02.055837 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d54bc49-39d1-499e-9c64-cff68590c793 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:55:02.075293 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1369/2754] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 17:55:01 2022] POST /identity/v3/users => generated 287 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:55:02.079471 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897c384a-a930-4045-87bd-e6a76298c22a admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:55:02.090094 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897c384a-a930-4045-87bd-e6a76298c22a admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:55:02.094655 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c728ca6a-1bca-41fc-bae5-3125df8988aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:55:02.095245 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c728ca6a-1bca-41fc-bae5-3125df8988aa admin admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:55:02.095522 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c728ca6a-1bca-41fc-bae5-3125df8988aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:55:02.095793 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c728ca6a-1bca-41fc-bae5-3125df8988aa admin admin] PATH_INFO: `/v3/roles` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:55:02.096459 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c728ca6a-1bca-41fc-bae5-3125df8988aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:55:02.098060 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c728ca6a-1bca-41fc-bae5-3125df8988aa admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:55:02.108189 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1386/2755] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:55:02 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:55:02.111966 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d54bc49-39d1-499e-9c64-cff68590c793 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:55:02.122031 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d54bc49-39d1-499e-9c64-cff68590c793 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:55:02.126587 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db03cabb-2ffe-4b34-aa0e-590fd41b8634 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:55:02.127183 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db03cabb-2ffe-4b34-aa0e-590fd41b8634 admin admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:55:02.127466 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db03cabb-2ffe-4b34-aa0e-590fd41b8634 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:55:02.127743 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db03cabb-2ffe-4b34-aa0e-590fd41b8634 admin admin] PATH_INFO: `/v3/roles` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:55:02.128384 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db03cabb-2ffe-4b34-aa0e-590fd41b8634 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:55:02.130039 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db03cabb-2ffe-4b34-aa0e-590fd41b8634 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:55:02.140764 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1370/2756] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:55:02 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:55:02.144713 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c728ca6a-1bca-41fc-bae5-3125df8988aa admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:55:02.155403 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c728ca6a-1bca-41fc-bae5-3125df8988aa admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:55:02.159945 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af5d236-d6a0-4d6a-bddb-9306b5a21804 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:55:02.160603 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af5d236-d6a0-4d6a-bddb-9306b5a21804 admin admin] REQUEST_METHOD: `PUT` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:55:02.160884 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af5d236-d6a0-4d6a-bddb-9306b5a21804 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:55:02.161156 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af5d236-d6a0-4d6a-bddb-9306b5a21804 admin admin] PATH_INFO: `/v3/projects/8c7f92175fd544b7b5060aba454b9efb/users/6f0cada7faf245888a777025deb8ed25/roles/02476a5b82eb4b188c18a5d706a3ef68` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:55:02.175492 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af5d236-d6a0-4d6a-bddb-9306b5a21804 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c7f92175fd544b7b5060aba454b9efb, user_id=6f0cada7faf245888a777025deb8ed25, role_id=02476a5b82eb4b188c18a5d706a3ef68)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:55:02.178167 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af5d236-d6a0-4d6a-bddb-9306b5a21804 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:55:02.190715 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1387/2757] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 17:55:02 2022] PUT /identity/v3/projects/8c7f92175fd544b7b5060aba454b9efb/users/6f0cada7faf245888a777025deb8ed25/roles/02476a5b82eb4b188c18a5d706a3ef68 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:55:02.200963 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1eddba-7c41-42c8-b1ae-b60219a852e6 None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:55:02.201254 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1eddba-7c41-42c8-b1ae-b60219a852e6 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:55:02.201532 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1eddba-7c41-42c8-b1ae-b60219a852e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:55:02.229280 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-ba1eddba-7c41-42c8-b1ae-b60219a852e6 None None] MFA Rules not processed for user `6f0cada7faf245888a777025deb8ed25`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:55:02.230485 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-ba1eddba-7c41-42c8-b1ae-b60219a852e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:55:02.294407 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1371/2758] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:55:02 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:55:02.298718 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af5d236-d6a0-4d6a-bddb-9306b5a21804 admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:55:02.364802 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af5d236-d6a0-4d6a-bddb-9306b5a21804 admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:55:02.369743 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d8b8c9-ce4c-422f-9970-6f13f6b21ae9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:55:02.370360 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d8b8c9-ce4c-422f-9970-6f13f6b21ae9 admin admin] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:55:02.370637 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d8b8c9-ce4c-422f-9970-6f13f6b21ae9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:55:02.371077 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d8b8c9-ce4c-422f-9970-6f13f6b21ae9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:55:02.371706 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d8b8c9-ce4c-422f-9970-6f13f6b21ae9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:55:02.378748 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d8b8c9-ce4c-422f-9970-6f13f6b21ae9 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:55:02.393897 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:55:02.408376 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1388/2759] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 17:55:02 2022] POST /identity/v3/projects => generated 373 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:55:02.412111 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1eddba-7c41-42c8-b1ae-b60219a852e6 None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:55:02.423557 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1eddba-7c41-42c8-b1ae-b60219a852e6 None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:55:02.479578 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6acafa-d4f8-4cb0-ac3d-b0352001a232 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:55:02.480189 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6acafa-d4f8-4cb0-ac3d-b0352001a232 admin admin] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:55:02.480471 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6acafa-d4f8-4cb0-ac3d-b0352001a232 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:55:02.480778 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6acafa-d4f8-4cb0-ac3d-b0352001a232 admin admin] PATH_INFO: `/v3/users` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:55:02.481397 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6acafa-d4f8-4cb0-ac3d-b0352001a232 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:55:02.483320 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6acafa-d4f8-4cb0-ac3d-b0352001a232 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:55:02.504421 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1372/2760] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 17:55:02 2022] POST /identity/v3/users => generated 293 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:55:02.508882 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d8b8c9-ce4c-422f-9970-6f13f6b21ae9 admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:55:02.519448 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d8b8c9-ce4c-422f-9970-6f13f6b21ae9 admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:55:02.524746 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17f3f74-c9d1-450b-be9b-9d7c0656bee1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:55:02.525351 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17f3f74-c9d1-450b-be9b-9d7c0656bee1 admin admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:55:02.525629 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17f3f74-c9d1-450b-be9b-9d7c0656bee1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:55:02.525903 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17f3f74-c9d1-450b-be9b-9d7c0656bee1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:55:02.526541 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17f3f74-c9d1-450b-be9b-9d7c0656bee1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:55:02.528167 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17f3f74-c9d1-450b-be9b-9d7c0656bee1 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:55:02.538705 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1389/2761] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:55:02 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:55:02.542786 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6acafa-d4f8-4cb0-ac3d-b0352001a232 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:55:02.553326 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6acafa-d4f8-4cb0-ac3d-b0352001a232 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:55:02.557903 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c07e194-f042-4d8e-a91d-19b38e9b42f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:55:02.558539 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c07e194-f042-4d8e-a91d-19b38e9b42f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:55:02.558815 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c07e194-f042-4d8e-a91d-19b38e9b42f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:55:02.559086 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c07e194-f042-4d8e-a91d-19b38e9b42f6 admin admin] PATH_INFO: `/v3/projects/8559713bf17b47dc90455e4aea3be4a5/users/1df34f99f3904e9db3d24c761a90f564/roles/e411ad9eb96e4e8d85b59b5b700ada7b` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:55:02.574036 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c07e194-f042-4d8e-a91d-19b38e9b42f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8559713bf17b47dc90455e4aea3be4a5, user_id=1df34f99f3904e9db3d24c761a90f564, role_id=e411ad9eb96e4e8d85b59b5b700ada7b)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:55:02.576856 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c07e194-f042-4d8e-a91d-19b38e9b42f6 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:55:02.593292 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1373/2762] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 17:55:02 2022] PUT /identity/v3/projects/8559713bf17b47dc90455e4aea3be4a5/users/1df34f99f3904e9db3d24c761a90f564/roles/e411ad9eb96e4e8d85b59b5b700ada7b => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:55:02.599061 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17f3f74-c9d1-450b-be9b-9d7c0656bee1 admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:55:02.608933 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17f3f74-c9d1-450b-be9b-9d7c0656bee1 admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:55:02.663177 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e7d428-f323-42ca-bf3f-9c8c68302b5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:55:02.663796 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e7d428-f323-42ca-bf3f-9c8c68302b5e admin admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:55:02.664072 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e7d428-f323-42ca-bf3f-9c8c68302b5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:55:02.664344 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e7d428-f323-42ca-bf3f-9c8c68302b5e admin admin] PATH_INFO: `/v3/roles` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:55:02.665073 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e7d428-f323-42ca-bf3f-9c8c68302b5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:55:02.666642 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e7d428-f323-42ca-bf3f-9c8c68302b5e admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:55:02.676472 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1390/2763] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:55:02 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:55:02.680511 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c07e194-f042-4d8e-a91d-19b38e9b42f6 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:55:02.693980 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c07e194-f042-4d8e-a91d-19b38e9b42f6 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:55:02.699164 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591776a9-f38f-4ad4-8bdf-79129c0dfd20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:55:02.699818 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591776a9-f38f-4ad4-8bdf-79129c0dfd20 admin admin] REQUEST_METHOD: `PUT` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:55:02.700123 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591776a9-f38f-4ad4-8bdf-79129c0dfd20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:55:02.700396 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591776a9-f38f-4ad4-8bdf-79129c0dfd20 admin admin] PATH_INFO: `/v3/projects/8559713bf17b47dc90455e4aea3be4a5/users/1df34f99f3904e9db3d24c761a90f564/roles/e411ad9eb96e4e8d85b59b5b700ada7b` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:55:02.702342 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591776a9-f38f-4ad4-8bdf-79129c0dfd20 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8559713bf17b47dc90455e4aea3be4a5, user_id=1df34f99f3904e9db3d24c761a90f564, role_id=e411ad9eb96e4e8d85b59b5b700ada7b)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:55:02.704995 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591776a9-f38f-4ad4-8bdf-79129c0dfd20 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:55:02.715613 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1374/2764] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 17:55:02 2022] PUT /identity/v3/projects/8559713bf17b47dc90455e4aea3be4a5/users/1df34f99f3904e9db3d24c761a90f564/roles/e411ad9eb96e4e8d85b59b5b700ada7b => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:55:02.726964 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005320a3-0854-4cd8-a7c8-730d5d7bc60e None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:55:02.727253 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005320a3-0854-4cd8-a7c8-730d5d7bc60e None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:55:02.727530 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005320a3-0854-4cd8-a7c8-730d5d7bc60e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:55:02.754347 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-005320a3-0854-4cd8-a7c8-730d5d7bc60e None None] MFA Rules not processed for user `1df34f99f3904e9db3d24c761a90f564`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:55:02.755562 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-005320a3-0854-4cd8-a7c8-730d5d7bc60e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:55:02.824877 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1391/2765] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:55:02 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:55:02.836008 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591776a9-f38f-4ad4-8bdf-79129c0dfd20 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:55:02.837744 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-591776a9-f38f-4ad4-8bdf-79129c0dfd20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:55:02.847503 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:55:02.896037 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591776a9-f38f-4ad4-8bdf-79129c0dfd20 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:55:02.899704 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae2dcca-7aa5-44b5-8301-7f837c1b39d1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59be342541264942ad87f60ff4de6c8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:55:02.900300 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae2dcca-7aa5-44b5-8301-7f837c1b39d1 service cinder] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:55:02.900613 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae2dcca-7aa5-44b5-8301-7f837c1b39d1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:55:02.900916 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae2dcca-7aa5-44b5-8301-7f837c1b39d1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:55:02.909655 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae2dcca-7aa5-44b5-8301-7f837c1b39d1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:55:02.911111 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae2dcca-7aa5-44b5-8301-7f837c1b39d1 service cinder] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:55:02.920411 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1375/2766] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:55:02 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:55:02.997995 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005320a3-0854-4cd8-a7c8-730d5d7bc60e None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:55:03.008027 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005320a3-0854-4cd8-a7c8-730d5d7bc60e None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:55:03.011612 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff4fe35-33c5-409a-b093-6ade423135b9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59be342541264942ad87f60ff4de6c8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:55:03.012204 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff4fe35-33c5-409a-b093-6ade423135b9 service cinder] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:55:03.012483 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff4fe35-33c5-409a-b093-6ade423135b9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:55:03.012799 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff4fe35-33c5-409a-b093-6ade423135b9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:55:03.067367 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff4fe35-33c5-409a-b093-6ade423135b9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:55:03.068892 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff4fe35-33c5-409a-b093-6ade423135b9 service cinder] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:55:03.078092 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1392/2767] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:55:02 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:55:12.916239 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=23 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:55:13.074177 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:INSERT=1,keystone:SELECT=98 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:55:46.364606 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae2dcca-7aa5-44b5-8301-7f837c1b39d1 service cinder] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:55:46.425805 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae2dcca-7aa5-44b5-8301-7f837c1b39d1 service cinder] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:55:46.430776 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04613378-c3bd-4ac8-b1dd-ab8d70728e53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:55:46.431394 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04613378-c3bd-4ac8-b1dd-ab8d70728e53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:55:46.431671 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04613378-c3bd-4ac8-b1dd-ab8d70728e53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:55:46.431975 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04613378-c3bd-4ac8-b1dd-ab8d70728e53 admin admin] PATH_INFO: `/v3/users/6f0cada7faf245888a777025deb8ed25` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:55:46.433103 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04613378-c3bd-4ac8-b1dd-ab8d70728e53 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6f0cada7faf245888a777025deb8ed25)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:55:46.435026 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04613378-c3bd-4ac8-b1dd-ab8d70728e53 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:55:46.480129 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-04613378-c3bd-4ac8-b1dd-ab8d70728e53 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6f0cada7faf245888a777025deb8ed25'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:55:46.480821 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-04613378-c3bd-4ac8-b1dd-ab8d70728e53 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6f0cada7faf245888a777025deb8ed25'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:55:46.487642 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-04613378-c3bd-4ac8-b1dd-ab8d70728e53 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6f0cada7faf245888a777025deb8ed25'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:55:46.488266 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-04613378-c3bd-4ac8-b1dd-ab8d70728e53 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6f0cada7faf245888a777025deb8ed25'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:55:46.495536 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-04613378-c3bd-4ac8-b1dd-ab8d70728e53 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6f0cada7faf245888a777025deb8ed25'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:55:46.513022 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1376/2768] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 17:55:46 2022] DELETE /identity/v3/users/6f0cada7faf245888a777025deb8ed25 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:55:46.517440 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4582e76f-f489-4af2-a316-e6d93460feac None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:55:46.517924 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4582e76f-f489-4af2-a316-e6d93460feac None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:55:46.518025 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4582e76f-f489-4af2-a316-e6d93460feac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:55:46.536131 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-4582e76f-f489-4af2-a316-e6d93460feac None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:55:46.537361 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-4582e76f-f489-4af2-a316-e6d93460feac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:55:46.594001 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1393/2769] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:55:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:55:46.604006 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04613378-c3bd-4ac8-b1dd-ab8d70728e53 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:55:46.605461 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-04613378-c3bd-4ac8-b1dd-ab8d70728e53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:55:46.661165 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04613378-c3bd-4ac8-b1dd-ab8d70728e53 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:55:46.665393 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654ac5a5-c452-4dd0-8f5f-822736743115 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43ad98a250a144099ccd18378aedd4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:55:46.665996 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654ac5a5-c452-4dd0-8f5f-822736743115 service neutron] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:55:46.666278 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654ac5a5-c452-4dd0-8f5f-822736743115 service neutron] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:55:46.666550 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654ac5a5-c452-4dd0-8f5f-822736743115 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:55:46.675440 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654ac5a5-c452-4dd0-8f5f-822736743115 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:55:46.676923 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654ac5a5-c452-4dd0-8f5f-822736743115 service neutron] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:55:46.686815 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1377/2770] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:55:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:55:46.950983 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4582e76f-f489-4af2-a316-e6d93460feac None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:55:46.952385 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-4582e76f-f489-4af2-a316-e6d93460feac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:55:46.965943 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4582e76f-f489-4af2-a316-e6d93460feac None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:55:46.970662 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be2b541-d9b5-40a5-95ef-d383f1c9263a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:55:46.971290 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be2b541-d9b5-40a5-95ef-d383f1c9263a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:55:46.971684 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be2b541-d9b5-40a5-95ef-d383f1c9263a admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:55:46.971964 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be2b541-d9b5-40a5-95ef-d383f1c9263a admin admin] PATH_INFO: `/v3/projects/8c7f92175fd544b7b5060aba454b9efb` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:55:46.973153 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be2b541-d9b5-40a5-95ef-d383f1c9263a admin admin] RBAC: Authorizing `identity:delete_project(project_id=8c7f92175fd544b7b5060aba454b9efb)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:55:46.975076 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be2b541-d9b5-40a5-95ef-d383f1c9263a admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:55:47.020111 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-8be2b541-d9b5-40a5-95ef-d383f1c9263a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8c7f92175fd544b7b5060aba454b9efb', 'request_id': 'req-8be2b541-d9b5-40a5-95ef-d383f1c9263a'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:55:47.027483 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-8be2b541-d9b5-40a5-95ef-d383f1c9263a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8c7f92175fd544b7b5060aba454b9efb', 'request_id': 'req-8be2b541-d9b5-40a5-95ef-d383f1c9263a'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:55:47.033554 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-8be2b541-d9b5-40a5-95ef-d383f1c9263a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8c7f92175fd544b7b5060aba454b9efb', 'request_id': 'req-8be2b541-d9b5-40a5-95ef-d383f1c9263a'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:55:47.044519 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-8be2b541-d9b5-40a5-95ef-d383f1c9263a admin admin] The token cache is being invalidate because project 8c7f92175fd544b7b5060aba454b9efb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81758) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:55:47.044841 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-8be2b541-d9b5-40a5-95ef-d383f1c9263a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:55:47.045463 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-8be2b541-d9b5-40a5-95ef-d383f1c9263a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:55:47.046911 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1394/2771] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 17:55:46 2022] DELETE /identity/v3/projects/8c7f92175fd544b7b5060aba454b9efb => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:55:47.050908 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654ac5a5-c452-4dd0-8f5f-822736743115 service neutron] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:55:47.060274 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654ac5a5-c452-4dd0-8f5f-822736743115 service neutron] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:55:47.077709 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:55:47.116953 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336eb5f3-c75a-423d-96b4-9aa0f4618185 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:55:47.117574 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336eb5f3-c75a-423d-96b4-9aa0f4618185 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:55:47.117850 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336eb5f3-c75a-423d-96b4-9aa0f4618185 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:55:47.118122 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336eb5f3-c75a-423d-96b4-9aa0f4618185 admin admin] PATH_INFO: `/v3/users/1df34f99f3904e9db3d24c761a90f564` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:55:47.119216 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336eb5f3-c75a-423d-96b4-9aa0f4618185 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1df34f99f3904e9db3d24c761a90f564)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:55:47.121197 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336eb5f3-c75a-423d-96b4-9aa0f4618185 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:55:47.166460 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-336eb5f3-c75a-423d-96b4-9aa0f4618185 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1df34f99f3904e9db3d24c761a90f564'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:55:47.167089 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-336eb5f3-c75a-423d-96b4-9aa0f4618185 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1df34f99f3904e9db3d24c761a90f564'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:55:47.173786 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-336eb5f3-c75a-423d-96b4-9aa0f4618185 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1df34f99f3904e9db3d24c761a90f564'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:55:47.174392 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-336eb5f3-c75a-423d-96b4-9aa0f4618185 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1df34f99f3904e9db3d24c761a90f564'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:55:47.181433 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-336eb5f3-c75a-423d-96b4-9aa0f4618185 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1df34f99f3904e9db3d24c761a90f564'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:55:47.198278 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1378/2772] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 17:55:47 2022] DELETE /identity/v3/users/1df34f99f3904e9db3d24c761a90f564 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:55:47.469079 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be2b541-d9b5-40a5-95ef-d383f1c9263a admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:55:47.470532 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-8be2b541-d9b5-40a5-95ef-d383f1c9263a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:55:47.486252 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be2b541-d9b5-40a5-95ef-d383f1c9263a admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:55:47.542909 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1144208-3c67-4ad0-be97-e1c2ad7002c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:55:47.543625 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1144208-3c67-4ad0-be97-e1c2ad7002c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:55:47.543802 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1144208-3c67-4ad0-be97-e1c2ad7002c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:55:47.544072 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1144208-3c67-4ad0-be97-e1c2ad7002c1 admin admin] PATH_INFO: `/v3/projects/8559713bf17b47dc90455e4aea3be4a5` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:55:47.545178 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1144208-3c67-4ad0-be97-e1c2ad7002c1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8559713bf17b47dc90455e4aea3be4a5)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:55:47.547057 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1144208-3c67-4ad0-be97-e1c2ad7002c1 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:55:47.576189 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:55:47.593860 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-e1144208-3c67-4ad0-be97-e1c2ad7002c1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8559713bf17b47dc90455e4aea3be4a5', 'request_id': 'req-e1144208-3c67-4ad0-be97-e1c2ad7002c1'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:55:47.602280 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-e1144208-3c67-4ad0-be97-e1c2ad7002c1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8559713bf17b47dc90455e4aea3be4a5', 'request_id': 'req-e1144208-3c67-4ad0-be97-e1c2ad7002c1'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:55:47.608622 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-e1144208-3c67-4ad0-be97-e1c2ad7002c1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8559713bf17b47dc90455e4aea3be4a5', 'request_id': 'req-e1144208-3c67-4ad0-be97-e1c2ad7002c1'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:55:47.618754 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-e1144208-3c67-4ad0-be97-e1c2ad7002c1 admin admin] The token cache is being invalidate because project 8559713bf17b47dc90455e4aea3be4a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81758) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:55:47.619042 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-e1144208-3c67-4ad0-be97-e1c2ad7002c1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:55:47.619663 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-e1144208-3c67-4ad0-be97-e1c2ad7002c1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:55:47.621141 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1395/2773] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 17:55:47 2022] DELETE /identity/v3/projects/8559713bf17b47dc90455e4aea3be4a5 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:55:47.626141 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06663faa-2bcb-4197-83d1-5969858e4d63 None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:55:47.626613 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06663faa-2bcb-4197-83d1-5969858e4d63 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:55:47.626727 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06663faa-2bcb-4197-83d1-5969858e4d63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:55:47.645942 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-06663faa-2bcb-4197-83d1-5969858e4d63 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:55:47.647637 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-06663faa-2bcb-4197-83d1-5969858e4d63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:55:47.704880 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1379/2774] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:55:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:55:47.713564 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6397f6e-6a7c-424b-b9b5-9b75d2abae7f None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:55:47.713788 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6397f6e-6a7c-424b-b9b5-9b75d2abae7f None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:55:47.714058 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6397f6e-6a7c-424b-b9b5-9b75d2abae7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:55:47.732999 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-e6397f6e-6a7c-424b-b9b5-9b75d2abae7f None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:55:47.734188 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-e6397f6e-6a7c-424b-b9b5-9b75d2abae7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:55:47.741848 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1396/2775] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:55:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:55:47.746021 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06663faa-2bcb-4197-83d1-5969858e4d63 None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:55:47.755623 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06663faa-2bcb-4197-83d1-5969858e4d63 None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:55:47.761010 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734bd841-75f1-4f55-a6e7-1f0a99308da6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:55:47.761609 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734bd841-75f1-4f55-a6e7-1f0a99308da6 admin admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:55:47.761892 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734bd841-75f1-4f55-a6e7-1f0a99308da6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:55:47.762169 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734bd841-75f1-4f55-a6e7-1f0a99308da6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:55:47.762815 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734bd841-75f1-4f55-a6e7-1f0a99308da6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:55:47.763118 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734bd841-75f1-4f55-a6e7-1f0a99308da6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:55:47.764739 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734bd841-75f1-4f55-a6e7-1f0a99308da6 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:55:47.776968 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1380/2776] 10.222.0.45 () {58 vars in 1153 bytes} [Wed Aug 24 17:55:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:55:47.781288 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6397f6e-6a7c-424b-b9b5-9b75d2abae7f None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:55:47.790492 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6397f6e-6a7c-424b-b9b5-9b75d2abae7f None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:55:47.795221 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab1d856-69ff-445e-a1f1-0078dfc4751f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:55:47.795817 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab1d856-69ff-445e-a1f1-0078dfc4751f admin admin] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:55:47.796097 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab1d856-69ff-445e-a1f1-0078dfc4751f admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:55:47.796387 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab1d856-69ff-445e-a1f1-0078dfc4751f admin admin] PATH_INFO: `/v3/projects` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:55:47.797039 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab1d856-69ff-445e-a1f1-0078dfc4751f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:55:47.799014 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab1d856-69ff-445e-a1f1-0078dfc4751f admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:55:47.826069 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1397/2777] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 17:55:47 2022] POST /identity/v3/projects => generated 401 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:55:47.830153 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734bd841-75f1-4f55-a6e7-1f0a99308da6 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:55:47.840620 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734bd841-75f1-4f55-a6e7-1f0a99308da6 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:55:47.865434 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:55:47.896642 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dea9f10-c545-49cb-9967-209fdee92013 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:55:47.897847 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dea9f10-c545-49cb-9967-209fdee92013 admin admin] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:55:47.898123 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dea9f10-c545-49cb-9967-209fdee92013 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:55:47.898390 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dea9f10-c545-49cb-9967-209fdee92013 admin admin] PATH_INFO: `/v3/users` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:55:47.899010 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dea9f10-c545-49cb-9967-209fdee92013 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:55:47.900973 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dea9f10-c545-49cb-9967-209fdee92013 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:55:47.922608 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1381/2778] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 17:55:47 2022] POST /identity/v3/users => generated 301 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:55:47.926570 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab1d856-69ff-445e-a1f1-0078dfc4751f admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:55:47.936452 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab1d856-69ff-445e-a1f1-0078dfc4751f admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:55:47.941033 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72afcfa7-cff4-4041-a94d-518a9443841a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:55:47.941622 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72afcfa7-cff4-4041-a94d-518a9443841a admin admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:55:47.941899 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72afcfa7-cff4-4041-a94d-518a9443841a admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:55:47.942169 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72afcfa7-cff4-4041-a94d-518a9443841a admin admin] PATH_INFO: `/v3/roles` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:55:47.942805 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72afcfa7-cff4-4041-a94d-518a9443841a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:55:47.944959 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72afcfa7-cff4-4041-a94d-518a9443841a admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:55:47.954566 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1398/2779] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:55:47 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:55:47.958245 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dea9f10-c545-49cb-9967-209fdee92013 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:55:47.968527 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dea9f10-c545-49cb-9967-209fdee92013 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:55:47.973010 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ad6f8e-55e6-44c8-826c-46730a20cff2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:55:47.973589 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ad6f8e-55e6-44c8-826c-46730a20cff2 admin admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:55:47.973867 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ad6f8e-55e6-44c8-826c-46730a20cff2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:55:47.974135 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ad6f8e-55e6-44c8-826c-46730a20cff2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:55:47.974761 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ad6f8e-55e6-44c8-826c-46730a20cff2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:55:47.976328 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ad6f8e-55e6-44c8-826c-46730a20cff2 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:55:47.986625 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1382/2780] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:55:47 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 17:55:47.990583 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72afcfa7-cff4-4041-a94d-518a9443841a admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:55:48.000349 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72afcfa7-cff4-4041-a94d-518a9443841a admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:55:48.005220 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306e1880-20be-42af-bb48-9f9be3fda502 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:55:48.005867 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306e1880-20be-42af-bb48-9f9be3fda502 admin admin] REQUEST_METHOD: `PUT` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:55:48.006150 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306e1880-20be-42af-bb48-9f9be3fda502 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:55:48.006943 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306e1880-20be-42af-bb48-9f9be3fda502 admin admin] PATH_INFO: `/v3/projects/dd32de4ae0884e6a803240de240f8cc8/users/d434ce95116b45c28fec0cf8f429817d/roles/02476a5b82eb4b188c18a5d706a3ef68` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:55:48.021510 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306e1880-20be-42af-bb48-9f9be3fda502 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd32de4ae0884e6a803240de240f8cc8, user_id=d434ce95116b45c28fec0cf8f429817d, role_id=02476a5b82eb4b188c18a5d706a3ef68)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:55:48.024796 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306e1880-20be-42af-bb48-9f9be3fda502 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:55:48.037100 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1399/2781] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 17:55:47 2022] PUT /identity/v3/projects/dd32de4ae0884e6a803240de240f8cc8/users/d434ce95116b45c28fec0cf8f429817d/roles/02476a5b82eb4b188c18a5d706a3ef68 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:55:48.100400 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4e55dd-7eea-402b-899b-337fccbd09a2 None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:55:48.100848 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4e55dd-7eea-402b-899b-337fccbd09a2 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:55:48.100999 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4e55dd-7eea-402b-899b-337fccbd09a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:55:48.128270 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-1c4e55dd-7eea-402b-899b-337fccbd09a2 None None] MFA Rules not processed for user `d434ce95116b45c28fec0cf8f429817d`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:55:48.129564 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-1c4e55dd-7eea-402b-899b-337fccbd09a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:55:48.191451 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1383/2782] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:55:48 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:55:48.196448 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306e1880-20be-42af-bb48-9f9be3fda502 admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:55:48.255595 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306e1880-20be-42af-bb48-9f9be3fda502 admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:55:48.260892 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107241fd-050d-4e83-8d3e-3f5bc36c06ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:55:48.261488 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107241fd-050d-4e83-8d3e-3f5bc36c06ae admin admin] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:55:48.261769 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107241fd-050d-4e83-8d3e-3f5bc36c06ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:55:48.262040 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107241fd-050d-4e83-8d3e-3f5bc36c06ae admin admin] PATH_INFO: `/v3/projects` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:55:48.262668 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107241fd-050d-4e83-8d3e-3f5bc36c06ae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:55:48.264609 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107241fd-050d-4e83-8d3e-3f5bc36c06ae admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:55:48.290666 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1400/2783] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 17:55:48 2022] POST /identity/v3/projects => generated 399 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:55:48.294836 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4e55dd-7eea-402b-899b-337fccbd09a2 None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:55:48.305320 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4e55dd-7eea-402b-899b-337fccbd09a2 None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:55:48.363640 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e8f49e-d5f9-461e-8f8d-c634a7d6afbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:55:48.364241 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e8f49e-d5f9-461e-8f8d-c634a7d6afbc admin admin] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:55:48.364515 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e8f49e-d5f9-461e-8f8d-c634a7d6afbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:55:48.364815 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e8f49e-d5f9-461e-8f8d-c634a7d6afbc admin admin] PATH_INFO: `/v3/users` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:55:48.365432 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e8f49e-d5f9-461e-8f8d-c634a7d6afbc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:55:48.367430 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e8f49e-d5f9-461e-8f8d-c634a7d6afbc admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:55:48.388083 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1384/2784] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 17:55:48 2022] POST /identity/v3/users => generated 306 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:55:48.392106 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107241fd-050d-4e83-8d3e-3f5bc36c06ae admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:55:48.402807 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107241fd-050d-4e83-8d3e-3f5bc36c06ae admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:55:48.407333 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e41be19-283e-47b7-87c0-d76b6cdb3ced admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:55:48.407924 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e41be19-283e-47b7-87c0-d76b6cdb3ced admin admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:55:48.408200 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e41be19-283e-47b7-87c0-d76b6cdb3ced admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:55:48.408474 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e41be19-283e-47b7-87c0-d76b6cdb3ced admin admin] PATH_INFO: `/v3/roles` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:55:48.409143 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e41be19-283e-47b7-87c0-d76b6cdb3ced admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:55:48.410715 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e41be19-283e-47b7-87c0-d76b6cdb3ced admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:55:48.421042 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1401/2785] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:55:48 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:55:48.424816 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e8f49e-d5f9-461e-8f8d-c634a7d6afbc admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:55:48.434540 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e8f49e-d5f9-461e-8f8d-c634a7d6afbc admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:55:48.439065 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52690fe-cd6a-4890-8634-e967611de71e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:55:48.439697 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52690fe-cd6a-4890-8634-e967611de71e admin admin] REQUEST_METHOD: `PUT` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:55:48.439975 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52690fe-cd6a-4890-8634-e967611de71e admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:55:48.440247 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52690fe-cd6a-4890-8634-e967611de71e admin admin] PATH_INFO: `/v3/projects/6e8cdbfdd2a34c578a20e3fd5acd6d96/users/ee1dfe4bde2c49ce8d8444f4bda75ab4/roles/e411ad9eb96e4e8d85b59b5b700ada7b` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:55:48.456107 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52690fe-cd6a-4890-8634-e967611de71e admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e8cdbfdd2a34c578a20e3fd5acd6d96, user_id=ee1dfe4bde2c49ce8d8444f4bda75ab4, role_id=e411ad9eb96e4e8d85b59b5b700ada7b)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:55:48.458763 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52690fe-cd6a-4890-8634-e967611de71e admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:55:48.472494 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1385/2786] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 17:55:48 2022] PUT /identity/v3/projects/6e8cdbfdd2a34c578a20e3fd5acd6d96/users/ee1dfe4bde2c49ce8d8444f4bda75ab4/roles/e411ad9eb96e4e8d85b59b5b700ada7b => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:55:48.476326 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e41be19-283e-47b7-87c0-d76b6cdb3ced admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:55:48.485937 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e41be19-283e-47b7-87c0-d76b6cdb3ced admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:55:48.506190 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:55:48.543925 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80d1988-8a1b-4ed2-900a-712d62ffc80e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:55:48.544589 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80d1988-8a1b-4ed2-900a-712d62ffc80e admin admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:55:48.544837 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80d1988-8a1b-4ed2-900a-712d62ffc80e admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:55:48.545113 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80d1988-8a1b-4ed2-900a-712d62ffc80e admin admin] PATH_INFO: `/v3/roles` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:55:48.545748 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80d1988-8a1b-4ed2-900a-712d62ffc80e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:55:48.547453 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80d1988-8a1b-4ed2-900a-712d62ffc80e admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:55:48.557339 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1402/2787] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:55:48 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:55:48.561424 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52690fe-cd6a-4890-8634-e967611de71e admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:55:48.566802 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:55:48.573444 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52690fe-cd6a-4890-8634-e967611de71e admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:55:48.578142 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1d3086-09ba-498b-91e4-e4256917bec7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:55:48.578788 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1d3086-09ba-498b-91e4-e4256917bec7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:55:48.579064 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1d3086-09ba-498b-91e4-e4256917bec7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:55:48.579333 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1d3086-09ba-498b-91e4-e4256917bec7 admin admin] PATH_INFO: `/v3/projects/6e8cdbfdd2a34c578a20e3fd5acd6d96/users/ee1dfe4bde2c49ce8d8444f4bda75ab4/roles/e411ad9eb96e4e8d85b59b5b700ada7b` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:55:48.581269 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1d3086-09ba-498b-91e4-e4256917bec7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e8cdbfdd2a34c578a20e3fd5acd6d96, user_id=ee1dfe4bde2c49ce8d8444f4bda75ab4, role_id=e411ad9eb96e4e8d85b59b5b700ada7b)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:55:48.583881 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1d3086-09ba-498b-91e4-e4256917bec7 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:55:48.593771 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1386/2788] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 17:55:48 2022] PUT /identity/v3/projects/6e8cdbfdd2a34c578a20e3fd5acd6d96/users/ee1dfe4bde2c49ce8d8444f4bda75ab4/roles/e411ad9eb96e4e8d85b59b5b700ada7b => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:55:48.603546 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f907f59-4dc1-49aa-a2e6-aac7ee1c40bf None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:55:48.603835 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f907f59-4dc1-49aa-a2e6-aac7ee1c40bf None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:55:48.604113 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f907f59-4dc1-49aa-a2e6-aac7ee1c40bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:55:48.630907 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-8f907f59-4dc1-49aa-a2e6-aac7ee1c40bf None None] MFA Rules not processed for user `ee1dfe4bde2c49ce8d8444f4bda75ab4`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:55:48.632155 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-8f907f59-4dc1-49aa-a2e6-aac7ee1c40bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:55:48.703725 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1403/2789] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:55:48 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:55:48.715092 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1d3086-09ba-498b-91e4-e4256917bec7 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:55:48.716820 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-ea1d3086-09ba-498b-91e4-e4256917bec7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:55:48.773907 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1d3086-09ba-498b-91e4-e4256917bec7 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:55:48.777503 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca227e87-efa6-4ec4-b30e-2d507ccc92e6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59be342541264942ad87f60ff4de6c8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:55:48.778138 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca227e87-efa6-4ec4-b30e-2d507ccc92e6 service cinder] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:55:48.778408 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca227e87-efa6-4ec4-b30e-2d507ccc92e6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:55:48.778674 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca227e87-efa6-4ec4-b30e-2d507ccc92e6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:55:48.788269 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca227e87-efa6-4ec4-b30e-2d507ccc92e6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:55:48.789719 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca227e87-efa6-4ec4-b30e-2d507ccc92e6 service cinder] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:55:48.798986 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1387/2790] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:55:48 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:55:48.874288 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f907f59-4dc1-49aa-a2e6-aac7ee1c40bf None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:55:48.884225 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f907f59-4dc1-49aa-a2e6-aac7ee1c40bf None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:55:48.888090 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7c757f-9bc7-41b1-8d1a-417b4f116a9f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59be342541264942ad87f60ff4de6c8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:55:48.888717 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7c757f-9bc7-41b1-8d1a-417b4f116a9f service cinder] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:55:48.888998 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7c757f-9bc7-41b1-8d1a-417b4f116a9f service cinder] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:55:48.889270 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7c757f-9bc7-41b1-8d1a-417b4f116a9f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:55:48.946200 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7c757f-9bc7-41b1-8d1a-417b4f116a9f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:55:48.947632 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7c757f-9bc7-41b1-8d1a-417b4f116a9f service cinder] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:55:48.956330 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1404/2791] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:55:48 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:55:53.000719 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca227e87-efa6-4ec4-b30e-2d507ccc92e6 service cinder] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:55:53.061606 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca227e87-efa6-4ec4-b30e-2d507ccc92e6 service cinder] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:55:53.066000 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b94e29-7170-426c-8daf-e0348e774b2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:55:53.066618 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b94e29-7170-426c-8daf-e0348e774b2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:55:53.066895 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b94e29-7170-426c-8daf-e0348e774b2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:55:53.067166 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b94e29-7170-426c-8daf-e0348e774b2c admin admin] PATH_INFO: `/v3/users/d434ce95116b45c28fec0cf8f429817d` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:55:53.068267 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b94e29-7170-426c-8daf-e0348e774b2c admin admin] RBAC: Authorizing `identity:delete_user(user_id=d434ce95116b45c28fec0cf8f429817d)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:55:53.070779 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b94e29-7170-426c-8daf-e0348e774b2c admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:55:53.115048 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-54b94e29-7170-426c-8daf-e0348e774b2c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd434ce95116b45c28fec0cf8f429817d'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:55:53.116177 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-54b94e29-7170-426c-8daf-e0348e774b2c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd434ce95116b45c28fec0cf8f429817d'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:55:53.122980 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-54b94e29-7170-426c-8daf-e0348e774b2c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd434ce95116b45c28fec0cf8f429817d'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:55:53.123588 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-54b94e29-7170-426c-8daf-e0348e774b2c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd434ce95116b45c28fec0cf8f429817d'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:55:53.131545 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-54b94e29-7170-426c-8daf-e0348e774b2c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd434ce95116b45c28fec0cf8f429817d'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:55:53.148418 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1388/2792] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 17:55:52 2022] DELETE /identity/v3/users/d434ce95116b45c28fec0cf8f429817d => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:55:53.153000 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40865353-a6df-44ff-926b-5786aaf9d376 None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:55:53.153499 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40865353-a6df-44ff-926b-5786aaf9d376 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:55:53.153714 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40865353-a6df-44ff-926b-5786aaf9d376 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:55:53.171923 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-40865353-a6df-44ff-926b-5786aaf9d376 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:55:53.173149 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-40865353-a6df-44ff-926b-5786aaf9d376 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:55:53.212187 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:55:53.230625 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1405/2793] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:55:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:55:53.240165 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b94e29-7170-426c-8daf-e0348e774b2c admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:55:53.242103 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-54b94e29-7170-426c-8daf-e0348e774b2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:55:53.255466 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:55:53.300341 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b94e29-7170-426c-8daf-e0348e774b2c admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:55:53.304459 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e171fa42-465a-4a3e-ba22-c19500a26678 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43ad98a250a144099ccd18378aedd4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:55:53.305088 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e171fa42-465a-4a3e-ba22-c19500a26678 service neutron] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:55:53.305366 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e171fa42-465a-4a3e-ba22-c19500a26678 service neutron] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:55:53.305639 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e171fa42-465a-4a3e-ba22-c19500a26678 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:55:53.314466 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e171fa42-465a-4a3e-ba22-c19500a26678 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:55:53.315888 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e171fa42-465a-4a3e-ba22-c19500a26678 service neutron] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:55:53.325770 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1389/2794] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:55:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:55:53.581243 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40865353-a6df-44ff-926b-5786aaf9d376 None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:55:53.582762 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-40865353-a6df-44ff-926b-5786aaf9d376 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:55:53.595635 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40865353-a6df-44ff-926b-5786aaf9d376 None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:55:53.600958 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e67220-d9ca-4011-9596-f62f5af325ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:55:53.601580 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e67220-d9ca-4011-9596-f62f5af325ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:55:53.601856 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e67220-d9ca-4011-9596-f62f5af325ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:55:53.602143 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e67220-d9ca-4011-9596-f62f5af325ab admin admin] PATH_INFO: `/v3/projects/dd32de4ae0884e6a803240de240f8cc8` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:55:53.603242 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e67220-d9ca-4011-9596-f62f5af325ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=dd32de4ae0884e6a803240de240f8cc8)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:55:53.605221 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e67220-d9ca-4011-9596-f62f5af325ab admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:55:53.651289 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-31e67220-d9ca-4011-9596-f62f5af325ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dd32de4ae0884e6a803240de240f8cc8', 'request_id': 'req-31e67220-d9ca-4011-9596-f62f5af325ab'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:55:53.658606 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-31e67220-d9ca-4011-9596-f62f5af325ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dd32de4ae0884e6a803240de240f8cc8', 'request_id': 'req-31e67220-d9ca-4011-9596-f62f5af325ab'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:55:53.665541 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-31e67220-d9ca-4011-9596-f62f5af325ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dd32de4ae0884e6a803240de240f8cc8', 'request_id': 'req-31e67220-d9ca-4011-9596-f62f5af325ab'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:55:53.675939 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-31e67220-d9ca-4011-9596-f62f5af325ab admin admin] The token cache is being invalidate because project dd32de4ae0884e6a803240de240f8cc8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81758) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:55:53.676228 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-31e67220-d9ca-4011-9596-f62f5af325ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:55:53.676879 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-31e67220-d9ca-4011-9596-f62f5af325ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:55:53.678986 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1406/2795] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 17:55:53 2022] DELETE /identity/v3/projects/dd32de4ae0884e6a803240de240f8cc8 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:55:53.682999 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e171fa42-465a-4a3e-ba22-c19500a26678 service neutron] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:55:53.692619 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e171fa42-465a-4a3e-ba22-c19500a26678 service neutron] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:55:53.748072 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e97b7a-8b59-4093-a1f3-4bdaf1dd6e9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:55:53.748738 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e97b7a-8b59-4093-a1f3-4bdaf1dd6e9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:55:53.749019 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e97b7a-8b59-4093-a1f3-4bdaf1dd6e9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:55:53.749291 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e97b7a-8b59-4093-a1f3-4bdaf1dd6e9b admin admin] PATH_INFO: `/v3/users/ee1dfe4bde2c49ce8d8444f4bda75ab4` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:55:53.750436 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e97b7a-8b59-4093-a1f3-4bdaf1dd6e9b admin admin] RBAC: Authorizing `identity:delete_user(user_id=ee1dfe4bde2c49ce8d8444f4bda75ab4)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:55:53.752399 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e97b7a-8b59-4093-a1f3-4bdaf1dd6e9b admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:55:53.798535 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-76e97b7a-8b59-4093-a1f3-4bdaf1dd6e9b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ee1dfe4bde2c49ce8d8444f4bda75ab4'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:55:53.799242 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-76e97b7a-8b59-4093-a1f3-4bdaf1dd6e9b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ee1dfe4bde2c49ce8d8444f4bda75ab4'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:55:53.806249 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-76e97b7a-8b59-4093-a1f3-4bdaf1dd6e9b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ee1dfe4bde2c49ce8d8444f4bda75ab4'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:55:53.806903 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-76e97b7a-8b59-4093-a1f3-4bdaf1dd6e9b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ee1dfe4bde2c49ce8d8444f4bda75ab4'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:55:53.813968 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-76e97b7a-8b59-4093-a1f3-4bdaf1dd6e9b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ee1dfe4bde2c49ce8d8444f4bda75ab4'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:55:53.830924 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1390/2796] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 17:55:53 2022] DELETE /identity/v3/users/ee1dfe4bde2c49ce8d8444f4bda75ab4 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:55:54.096470 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e67220-d9ca-4011-9596-f62f5af325ab admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:55:54.097926 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-31e67220-d9ca-4011-9596-f62f5af325ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:55:54.110921 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e67220-d9ca-4011-9596-f62f5af325ab admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:55:54.167845 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fd81f0-c6bc-4cd4-b1c1-d87fa2a8af7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:55:54.168575 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fd81f0-c6bc-4cd4-b1c1-d87fa2a8af7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:55:54.168787 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fd81f0-c6bc-4cd4-b1c1-d87fa2a8af7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:55:54.169064 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fd81f0-c6bc-4cd4-b1c1-d87fa2a8af7e admin admin] PATH_INFO: `/v3/projects/6e8cdbfdd2a34c578a20e3fd5acd6d96` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:55:54.170188 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fd81f0-c6bc-4cd4-b1c1-d87fa2a8af7e admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e8cdbfdd2a34c578a20e3fd5acd6d96)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:55:54.172095 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fd81f0-c6bc-4cd4-b1c1-d87fa2a8af7e admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:55:54.218582 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-08fd81f0-c6bc-4cd4-b1c1-d87fa2a8af7e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e8cdbfdd2a34c578a20e3fd5acd6d96', 'request_id': 'req-08fd81f0-c6bc-4cd4-b1c1-d87fa2a8af7e'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:55:54.225600 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-08fd81f0-c6bc-4cd4-b1c1-d87fa2a8af7e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e8cdbfdd2a34c578a20e3fd5acd6d96', 'request_id': 'req-08fd81f0-c6bc-4cd4-b1c1-d87fa2a8af7e'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:55:54.231624 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-08fd81f0-c6bc-4cd4-b1c1-d87fa2a8af7e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e8cdbfdd2a34c578a20e3fd5acd6d96', 'request_id': 'req-08fd81f0-c6bc-4cd4-b1c1-d87fa2a8af7e'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:55:54.238068 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:55:54.242939 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-08fd81f0-c6bc-4cd4-b1c1-d87fa2a8af7e admin admin] The token cache is being invalidate because project 6e8cdbfdd2a34c578a20e3fd5acd6d96 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81758) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:55:54.243441 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-08fd81f0-c6bc-4cd4-b1c1-d87fa2a8af7e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:55:54.244614 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-08fd81f0-c6bc-4cd4-b1c1-d87fa2a8af7e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:55:54.246169 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1407/2797] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 17:55:54 2022] DELETE /identity/v3/projects/6e8cdbfdd2a34c578a20e3fd5acd6d96 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:55:54.251022 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201e9676-7e1b-42fb-b839-0c0b51ddfec4 None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:55:54.251485 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201e9676-7e1b-42fb-b839-0c0b51ddfec4 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:55:54.251583 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201e9676-7e1b-42fb-b839-0c0b51ddfec4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:55:54.271348 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-201e9676-7e1b-42fb-b839-0c0b51ddfec4 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:55:54.273141 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-201e9676-7e1b-42fb-b839-0c0b51ddfec4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:55:54.301020 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:55:54.332738 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1391/2798] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:55:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:55:54.342439 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2e45a1-5fea-4279-9687-c13d23046ca3 None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:55:54.342666 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2e45a1-5fea-4279-9687-c13d23046ca3 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:55:54.342943 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2e45a1-5fea-4279-9687-c13d23046ca3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:55:54.361877 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-0f2e45a1-5fea-4279-9687-c13d23046ca3 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:55:54.363075 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-0f2e45a1-5fea-4279-9687-c13d23046ca3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:55:54.370822 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1408/2799] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:55:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:55:54.374840 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201e9676-7e1b-42fb-b839-0c0b51ddfec4 None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:55:54.385380 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201e9676-7e1b-42fb-b839-0c0b51ddfec4 None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:55:54.390186 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18620199-96f7-4d72-b7c2-15ba2a99b88a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:55:54.390792 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18620199-96f7-4d72-b7c2-15ba2a99b88a admin admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:55:54.391074 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18620199-96f7-4d72-b7c2-15ba2a99b88a admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:55:54.391350 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18620199-96f7-4d72-b7c2-15ba2a99b88a admin admin] PATH_INFO: `/v3/domains` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:55:54.391997 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18620199-96f7-4d72-b7c2-15ba2a99b88a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:55:54.392303 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18620199-96f7-4d72-b7c2-15ba2a99b88a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:55:54.394008 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18620199-96f7-4d72-b7c2-15ba2a99b88a admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:55:54.406107 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1392/2800] 10.222.0.45 () {58 vars in 1153 bytes} [Wed Aug 24 17:55:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:55:54.410045 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2e45a1-5fea-4279-9687-c13d23046ca3 None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:55:54.419042 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2e45a1-5fea-4279-9687-c13d23046ca3 None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:55:54.423632 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94996132-343f-40b7-b65e-783498e6fdb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:55:54.424216 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94996132-343f-40b7-b65e-783498e6fdb6 admin admin] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:55:54.424492 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94996132-343f-40b7-b65e-783498e6fdb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:55:54.424790 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94996132-343f-40b7-b65e-783498e6fdb6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:55:54.425403 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94996132-343f-40b7-b65e-783498e6fdb6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:55:54.427280 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94996132-343f-40b7-b65e-783498e6fdb6 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:55:54.453378 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1409/2801] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 17:55:54 2022] POST /identity/v3/projects => generated 381 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:55:54.457522 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18620199-96f7-4d72-b7c2-15ba2a99b88a admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:55:54.467917 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18620199-96f7-4d72-b7c2-15ba2a99b88a admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:55:54.522778 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b0bddf-0f76-4850-91c7-eec2c4920b2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:55:54.523374 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b0bddf-0f76-4850-91c7-eec2c4920b2a admin admin] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:55:54.523654 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b0bddf-0f76-4850-91c7-eec2c4920b2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:55:54.523928 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b0bddf-0f76-4850-91c7-eec2c4920b2a admin admin] PATH_INFO: `/v3/users` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:55:54.524538 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b0bddf-0f76-4850-91c7-eec2c4920b2a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:55:54.526491 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b0bddf-0f76-4850-91c7-eec2c4920b2a admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:55:54.546362 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1393/2802] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 17:55:54 2022] POST /identity/v3/users => generated 291 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:55:54.550526 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94996132-343f-40b7-b65e-783498e6fdb6 admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:55:54.561178 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94996132-343f-40b7-b65e-783498e6fdb6 admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:55:54.565731 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613c5bf3-14c7-402a-964c-065690896b2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:55:54.566330 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613c5bf3-14c7-402a-964c-065690896b2d admin admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:55:54.566611 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613c5bf3-14c7-402a-964c-065690896b2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:55:54.566886 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613c5bf3-14c7-402a-964c-065690896b2d admin admin] PATH_INFO: `/v3/roles` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:55:54.567529 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613c5bf3-14c7-402a-964c-065690896b2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:55:54.569762 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613c5bf3-14c7-402a-964c-065690896b2d admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:55:54.579431 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1410/2803] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:55:54 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:55:54.583122 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b0bddf-0f76-4850-91c7-eec2c4920b2a admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:55:54.593372 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b0bddf-0f76-4850-91c7-eec2c4920b2a admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:55:54.597999 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d648135-e3bf-452c-afca-9e21fc0946da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:55:54.598590 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d648135-e3bf-452c-afca-9e21fc0946da admin admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:55:54.598872 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d648135-e3bf-452c-afca-9e21fc0946da admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:55:54.599146 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d648135-e3bf-452c-afca-9e21fc0946da admin admin] PATH_INFO: `/v3/roles` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:55:54.599808 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d648135-e3bf-452c-afca-9e21fc0946da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:55:54.601439 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d648135-e3bf-452c-afca-9e21fc0946da admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:55:54.612269 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1394/2804] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:55:54 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:55:54.616854 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613c5bf3-14c7-402a-964c-065690896b2d admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:55:54.626501 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613c5bf3-14c7-402a-964c-065690896b2d admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:55:54.631866 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686df42f-1bb8-4b48-b1c8-e46927986651 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:55:54.632503 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686df42f-1bb8-4b48-b1c8-e46927986651 admin admin] REQUEST_METHOD: `PUT` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:55:54.632811 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686df42f-1bb8-4b48-b1c8-e46927986651 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:55:54.633083 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686df42f-1bb8-4b48-b1c8-e46927986651 admin admin] PATH_INFO: `/v3/projects/0b58cd50217e4b30a516c33e962b535e/users/3e2fb183409a4431b79ec1f2dc7b680d/roles/02476a5b82eb4b188c18a5d706a3ef68` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:55:54.647361 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686df42f-1bb8-4b48-b1c8-e46927986651 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b58cd50217e4b30a516c33e962b535e, user_id=3e2fb183409a4431b79ec1f2dc7b680d, role_id=02476a5b82eb4b188c18a5d706a3ef68)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:55:54.650032 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686df42f-1bb8-4b48-b1c8-e46927986651 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:55:54.662814 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1411/2805] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 17:55:54 2022] PUT /identity/v3/projects/0b58cd50217e4b30a516c33e962b535e/users/3e2fb183409a4431b79ec1f2dc7b680d/roles/02476a5b82eb4b188c18a5d706a3ef68 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:55:54.673032 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45150235-1303-4a5d-a09b-ad47bdd9b4eb None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:55:54.673349 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45150235-1303-4a5d-a09b-ad47bdd9b4eb None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:55:54.673655 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45150235-1303-4a5d-a09b-ad47bdd9b4eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:55:54.702950 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-45150235-1303-4a5d-a09b-ad47bdd9b4eb None None] MFA Rules not processed for user `3e2fb183409a4431b79ec1f2dc7b680d`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:55:54.704198 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-45150235-1303-4a5d-a09b-ad47bdd9b4eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:55:54.766917 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1395/2806] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:55:54 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:55:54.771427 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686df42f-1bb8-4b48-b1c8-e46927986651 admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:55:54.832872 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686df42f-1bb8-4b48-b1c8-e46927986651 admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:55:54.837418 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb092c84-edc6-4098-80fa-7485e66789ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:55:54.838010 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb092c84-edc6-4098-80fa-7485e66789ab admin admin] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:55:54.838282 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb092c84-edc6-4098-80fa-7485e66789ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:55:54.838549 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb092c84-edc6-4098-80fa-7485e66789ab admin admin] PATH_INFO: `/v3/projects` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:55:54.839155 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb092c84-edc6-4098-80fa-7485e66789ab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:55:54.841051 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb092c84-edc6-4098-80fa-7485e66789ab admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:55:54.867899 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1412/2807] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 17:55:54 2022] POST /identity/v3/projects => generated 381 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:55:54.871883 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45150235-1303-4a5d-a09b-ad47bdd9b4eb None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:55:54.882725 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45150235-1303-4a5d-a09b-ad47bdd9b4eb None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:55:54.908305 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:55:54.939834 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bcae62-ce83-49cd-adfc-65007a5f9152 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:55:54.941454 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bcae62-ce83-49cd-adfc-65007a5f9152 admin admin] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:55:54.941736 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bcae62-ce83-49cd-adfc-65007a5f9152 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:55:54.942046 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bcae62-ce83-49cd-adfc-65007a5f9152 admin admin] PATH_INFO: `/v3/users` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:55:54.942699 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bcae62-ce83-49cd-adfc-65007a5f9152 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:55:54.944668 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bcae62-ce83-49cd-adfc-65007a5f9152 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:55:54.968934 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1396/2808] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 17:55:54 2022] POST /identity/v3/users => generated 297 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:55:54.970065 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb092c84-edc6-4098-80fa-7485e66789ab admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:55:54.980685 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb092c84-edc6-4098-80fa-7485e66789ab admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:55:54.985555 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0281db9-3779-489f-900b-a603ba2e9840 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:55:54.986153 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0281db9-3779-489f-900b-a603ba2e9840 admin admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:55:54.986432 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0281db9-3779-489f-900b-a603ba2e9840 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:55:54.986704 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0281db9-3779-489f-900b-a603ba2e9840 admin admin] PATH_INFO: `/v3/roles` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:55:54.987345 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0281db9-3779-489f-900b-a603ba2e9840 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:55:54.988967 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0281db9-3779-489f-900b-a603ba2e9840 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:55:55.000277 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1413/2809] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:55:54 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 17:55:55.004113 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bcae62-ce83-49cd-adfc-65007a5f9152 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:55:55.014033 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bcae62-ce83-49cd-adfc-65007a5f9152 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:55:55.018562 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6ef029-f6a6-4ca2-a110-b69005ab9f2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:55:55.019196 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6ef029-f6a6-4ca2-a110-b69005ab9f2f admin admin] REQUEST_METHOD: `PUT` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:55:55.019488 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6ef029-f6a6-4ca2-a110-b69005ab9f2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:55:55.019747 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6ef029-f6a6-4ca2-a110-b69005ab9f2f admin admin] PATH_INFO: `/v3/projects/372d93ceb8db41019a230effc17912ec/users/b6a2acbd5c574d74896ffcf7b9984d0d/roles/e411ad9eb96e4e8d85b59b5b700ada7b` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:55:55.033920 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6ef029-f6a6-4ca2-a110-b69005ab9f2f admin admin] RBAC: Authorizing `identity:create_grant(project_id=372d93ceb8db41019a230effc17912ec, user_id=b6a2acbd5c574d74896ffcf7b9984d0d, role_id=e411ad9eb96e4e8d85b59b5b700ada7b)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:55:55.036538 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6ef029-f6a6-4ca2-a110-b69005ab9f2f admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:55:55.049804 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1397/2810] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 17:55:55 2022] PUT /identity/v3/projects/372d93ceb8db41019a230effc17912ec/users/b6a2acbd5c574d74896ffcf7b9984d0d/roles/e411ad9eb96e4e8d85b59b5b700ada7b => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:55:55.054293 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0281db9-3779-489f-900b-a603ba2e9840 admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:55:55.063974 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0281db9-3779-489f-900b-a603ba2e9840 admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:55:55.117772 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eeed53f-97c0-4439-84eb-15550dd69154 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:55:55.118427 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eeed53f-97c0-4439-84eb-15550dd69154 admin admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:55:55.118657 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eeed53f-97c0-4439-84eb-15550dd69154 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:55:55.118932 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eeed53f-97c0-4439-84eb-15550dd69154 admin admin] PATH_INFO: `/v3/roles` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:55:55.119568 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eeed53f-97c0-4439-84eb-15550dd69154 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:55:55.121187 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eeed53f-97c0-4439-84eb-15550dd69154 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:55:55.130591 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1414/2811] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:55:55 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:55:55.134615 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6ef029-f6a6-4ca2-a110-b69005ab9f2f admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:55:55.145105 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6ef029-f6a6-4ca2-a110-b69005ab9f2f admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:55:55.149772 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba57396-547d-4b0b-8ddc-e995138105c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:55:55.150418 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba57396-547d-4b0b-8ddc-e995138105c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:55:55.150692 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba57396-547d-4b0b-8ddc-e995138105c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:55:55.150981 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba57396-547d-4b0b-8ddc-e995138105c5 admin admin] PATH_INFO: `/v3/projects/372d93ceb8db41019a230effc17912ec/users/b6a2acbd5c574d74896ffcf7b9984d0d/roles/e411ad9eb96e4e8d85b59b5b700ada7b` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:55:55.152922 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba57396-547d-4b0b-8ddc-e995138105c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=372d93ceb8db41019a230effc17912ec, user_id=b6a2acbd5c574d74896ffcf7b9984d0d, role_id=e411ad9eb96e4e8d85b59b5b700ada7b)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:55:55.155580 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba57396-547d-4b0b-8ddc-e995138105c5 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:55:55.166157 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1398/2812] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 17:55:55 2022] PUT /identity/v3/projects/372d93ceb8db41019a230effc17912ec/users/b6a2acbd5c574d74896ffcf7b9984d0d/roles/e411ad9eb96e4e8d85b59b5b700ada7b => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:55:55.176272 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea97b625-0d75-4484-a735-7411c0a883d8 None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:55:55.176794 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea97b625-0d75-4484-a735-7411c0a883d8 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:55:55.176890 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea97b625-0d75-4484-a735-7411c0a883d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:55:55.185267 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:55:55.204820 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-ea97b625-0d75-4484-a735-7411c0a883d8 None None] MFA Rules not processed for user `b6a2acbd5c574d74896ffcf7b9984d0d`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:55:55.206015 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-ea97b625-0d75-4484-a735-7411c0a883d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:55:55.271835 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1415/2813] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:55:55 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:55:55.282932 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba57396-547d-4b0b-8ddc-e995138105c5 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:55:55.284666 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-bba57396-547d-4b0b-8ddc-e995138105c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:55:55.342468 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba57396-547d-4b0b-8ddc-e995138105c5 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:55:55.346169 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f3a9a3-2d03-430c-9aa3-9830cc7d8b0f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59be342541264942ad87f60ff4de6c8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:55:55.346765 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f3a9a3-2d03-430c-9aa3-9830cc7d8b0f service cinder] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:55:55.347040 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f3a9a3-2d03-430c-9aa3-9830cc7d8b0f service cinder] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:55:55.347310 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f3a9a3-2d03-430c-9aa3-9830cc7d8b0f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:55:55.356688 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f3a9a3-2d03-430c-9aa3-9830cc7d8b0f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:55:55.358105 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f3a9a3-2d03-430c-9aa3-9830cc7d8b0f service cinder] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:55:55.367229 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1399/2814] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:55:55 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:55:55.466388 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea97b625-0d75-4484-a735-7411c0a883d8 None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:55:55.476872 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea97b625-0d75-4484-a735-7411c0a883d8 None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:55:55.480374 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544f5534-4f3f-4463-a646-70feff8b7e49 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59be342541264942ad87f60ff4de6c8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:55:55.481001 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544f5534-4f3f-4463-a646-70feff8b7e49 service cinder] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:55:55.481304 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544f5534-4f3f-4463-a646-70feff8b7e49 service cinder] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:55:55.481572 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544f5534-4f3f-4463-a646-70feff8b7e49 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:55:55.535685 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544f5534-4f3f-4463-a646-70feff8b7e49 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:55:55.537143 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544f5534-4f3f-4463-a646-70feff8b7e49 service cinder] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:55:55.546350 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1416/2815] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:55:55 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:55:58.117176 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f3a9a3-2d03-430c-9aa3-9830cc7d8b0f service cinder] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:55:58.177197 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f3a9a3-2d03-430c-9aa3-9830cc7d8b0f service cinder] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:55:58.182149 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554404b5-c9fd-47d0-ad95-ff5c99037eed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:55:58.182789 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554404b5-c9fd-47d0-ad95-ff5c99037eed admin admin] REQUEST_METHOD: `DELETE` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:55:58.183066 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554404b5-c9fd-47d0-ad95-ff5c99037eed admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:55:58.183335 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554404b5-c9fd-47d0-ad95-ff5c99037eed admin admin] PATH_INFO: `/v3/users/3e2fb183409a4431b79ec1f2dc7b680d` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:55:58.184427 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554404b5-c9fd-47d0-ad95-ff5c99037eed admin admin] RBAC: Authorizing `identity:delete_user(user_id=3e2fb183409a4431b79ec1f2dc7b680d)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:55:58.186365 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554404b5-c9fd-47d0-ad95-ff5c99037eed admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:55:58.213390 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=91,keystone:INSERT=5,keystone:DELETE=4 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:55:58.233471 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-554404b5-c9fd-47d0-ad95-ff5c99037eed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3e2fb183409a4431b79ec1f2dc7b680d'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:55:58.234107 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-554404b5-c9fd-47d0-ad95-ff5c99037eed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3e2fb183409a4431b79ec1f2dc7b680d'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:55:58.240944 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-554404b5-c9fd-47d0-ad95-ff5c99037eed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3e2fb183409a4431b79ec1f2dc7b680d'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:55:58.241564 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-554404b5-c9fd-47d0-ad95-ff5c99037eed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3e2fb183409a4431b79ec1f2dc7b680d'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:55:58.248701 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-554404b5-c9fd-47d0-ad95-ff5c99037eed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3e2fb183409a4431b79ec1f2dc7b680d'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:55:58.265812 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1400/2816] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 17:55:58 2022] DELETE /identity/v3/users/3e2fb183409a4431b79ec1f2dc7b680d => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:55:58.270213 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bb40a6-0605-4eb5-a1ff-0152ff28ee80 None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:55:58.270696 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bb40a6-0605-4eb5-a1ff-0152ff28ee80 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:55:58.270818 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bb40a6-0605-4eb5-a1ff-0152ff28ee80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:55:58.290409 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-c3bb40a6-0605-4eb5-a1ff-0152ff28ee80 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:55:58.291606 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-c3bb40a6-0605-4eb5-a1ff-0152ff28ee80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:55:58.349779 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1417/2817] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:55:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:55:58.360117 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554404b5-c9fd-47d0-ad95-ff5c99037eed admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:55:58.361550 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-554404b5-c9fd-47d0-ad95-ff5c99037eed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:55:58.418045 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554404b5-c9fd-47d0-ad95-ff5c99037eed admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:55:58.421758 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225193a2-2f0f-443b-9088-87219cd8e40d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43ad98a250a144099ccd18378aedd4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:55:58.422438 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225193a2-2f0f-443b-9088-87219cd8e40d service neutron] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:55:58.422715 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225193a2-2f0f-443b-9088-87219cd8e40d service neutron] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:55:58.422985 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225193a2-2f0f-443b-9088-87219cd8e40d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:55:58.431722 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225193a2-2f0f-443b-9088-87219cd8e40d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:55:58.433161 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225193a2-2f0f-443b-9088-87219cd8e40d service neutron] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:55:58.442662 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1401/2818] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:55:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:55:58.714158 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bb40a6-0605-4eb5-a1ff-0152ff28ee80 None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:55:58.715655 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-c3bb40a6-0605-4eb5-a1ff-0152ff28ee80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:55:58.729262 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bb40a6-0605-4eb5-a1ff-0152ff28ee80 None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:55:58.733943 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ffccbf-f6de-4fe0-acfc-c3e1961aec54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:55:58.734574 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ffccbf-f6de-4fe0-acfc-c3e1961aec54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:55:58.734856 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ffccbf-f6de-4fe0-acfc-c3e1961aec54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:55:58.735250 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ffccbf-f6de-4fe0-acfc-c3e1961aec54 admin admin] PATH_INFO: `/v3/projects/0b58cd50217e4b30a516c33e962b535e` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:55:58.736383 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ffccbf-f6de-4fe0-acfc-c3e1961aec54 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b58cd50217e4b30a516c33e962b535e)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:55:58.738440 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ffccbf-f6de-4fe0-acfc-c3e1961aec54 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:55:58.763739 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:55:58.787622 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-58ffccbf-f6de-4fe0-acfc-c3e1961aec54 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b58cd50217e4b30a516c33e962b535e', 'request_id': 'req-58ffccbf-f6de-4fe0-acfc-c3e1961aec54'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:55:58.795343 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-58ffccbf-f6de-4fe0-acfc-c3e1961aec54 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b58cd50217e4b30a516c33e962b535e', 'request_id': 'req-58ffccbf-f6de-4fe0-acfc-c3e1961aec54'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:55:58.802072 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-58ffccbf-f6de-4fe0-acfc-c3e1961aec54 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b58cd50217e4b30a516c33e962b535e', 'request_id': 'req-58ffccbf-f6de-4fe0-acfc-c3e1961aec54'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:55:58.812351 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-58ffccbf-f6de-4fe0-acfc-c3e1961aec54 admin admin] The token cache is being invalidate because project 0b58cd50217e4b30a516c33e962b535e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81758) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:55:58.812678 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-58ffccbf-f6de-4fe0-acfc-c3e1961aec54 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:55:58.813308 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-58ffccbf-f6de-4fe0-acfc-c3e1961aec54 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:55:58.814768 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1418/2819] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 17:55:58 2022] DELETE /identity/v3/projects/0b58cd50217e4b30a516c33e962b535e => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:55:58.819339 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225193a2-2f0f-443b-9088-87219cd8e40d service neutron] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:55:58.829031 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225193a2-2f0f-443b-9088-87219cd8e40d service neutron] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:55:58.884349 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dc436a-e660-4cfc-b73c-42fbbc3dd831 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:55:58.885263 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dc436a-e660-4cfc-b73c-42fbbc3dd831 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:55:58.885543 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dc436a-e660-4cfc-b73c-42fbbc3dd831 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:55:58.885817 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dc436a-e660-4cfc-b73c-42fbbc3dd831 admin admin] PATH_INFO: `/v3/users/b6a2acbd5c574d74896ffcf7b9984d0d` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:55:58.886927 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22dc436a-e660-4cfc-b73c-42fbbc3dd831 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b6a2acbd5c574d74896ffcf7b9984d0d)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:55:58.889076 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22dc436a-e660-4cfc-b73c-42fbbc3dd831 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:55:58.938879 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-22dc436a-e660-4cfc-b73c-42fbbc3dd831 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b6a2acbd5c574d74896ffcf7b9984d0d'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:55:58.939508 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-22dc436a-e660-4cfc-b73c-42fbbc3dd831 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b6a2acbd5c574d74896ffcf7b9984d0d'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:55:58.940703 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:55:58.950217 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-22dc436a-e660-4cfc-b73c-42fbbc3dd831 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b6a2acbd5c574d74896ffcf7b9984d0d'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:55:58.950970 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-22dc436a-e660-4cfc-b73c-42fbbc3dd831 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b6a2acbd5c574d74896ffcf7b9984d0d'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:55:58.958095 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-22dc436a-e660-4cfc-b73c-42fbbc3dd831 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b6a2acbd5c574d74896ffcf7b9984d0d'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:55:58.975190 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1402/2820] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 17:55:58 2022] DELETE /identity/v3/users/b6a2acbd5c574d74896ffcf7b9984d0d => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:55:59.237590 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ffccbf-f6de-4fe0-acfc-c3e1961aec54 admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:55:59.239169 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-58ffccbf-f6de-4fe0-acfc-c3e1961aec54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:55:59.253111 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ffccbf-f6de-4fe0-acfc-c3e1961aec54 admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:55:59.309977 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c04768-027a-4280-b076-a61d05ce5cf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:55:59.310649 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c04768-027a-4280-b076-a61d05ce5cf5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:55:59.310897 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c04768-027a-4280-b076-a61d05ce5cf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:55:59.311176 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c04768-027a-4280-b076-a61d05ce5cf5 admin admin] PATH_INFO: `/v3/projects/372d93ceb8db41019a230effc17912ec` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:55:59.312372 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c04768-027a-4280-b076-a61d05ce5cf5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=372d93ceb8db41019a230effc17912ec)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:55:59.314310 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c04768-027a-4280-b076-a61d05ce5cf5 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:55:59.359545 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-20c04768-027a-4280-b076-a61d05ce5cf5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '372d93ceb8db41019a230effc17912ec', 'request_id': 'req-20c04768-027a-4280-b076-a61d05ce5cf5'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:55:59.367075 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-20c04768-027a-4280-b076-a61d05ce5cf5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '372d93ceb8db41019a230effc17912ec', 'request_id': 'req-20c04768-027a-4280-b076-a61d05ce5cf5'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:55:59.373081 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-20c04768-027a-4280-b076-a61d05ce5cf5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '372d93ceb8db41019a230effc17912ec', 'request_id': 'req-20c04768-027a-4280-b076-a61d05ce5cf5'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:55:59.383180 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-20c04768-027a-4280-b076-a61d05ce5cf5 admin admin] The token cache is being invalidate because project 372d93ceb8db41019a230effc17912ec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81758) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:55:59.383466 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-20c04768-027a-4280-b076-a61d05ce5cf5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:55:59.384081 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-20c04768-027a-4280-b076-a61d05ce5cf5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:55:59.385825 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1419/2821] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 17:55:59 2022] DELETE /identity/v3/projects/372d93ceb8db41019a230effc17912ec => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:55:59.390664 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd97a44-b5bb-4b3b-b3ef-e2c3e2ef99ec None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:55:59.391171 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd97a44-b5bb-4b3b-b3ef-e2c3e2ef99ec None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:55:59.391277 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd97a44-b5bb-4b3b-b3ef-e2c3e2ef99ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:55:59.410190 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-0fd97a44-b5bb-4b3b-b3ef-e2c3e2ef99ec None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:55:59.411377 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-0fd97a44-b5bb-4b3b-b3ef-e2c3e2ef99ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:55:59.469452 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1403/2822] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:55:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:55:59.477627 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373d1d23-2a5e-4cd5-ab4c-3f38054041d2 None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:55:59.477914 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373d1d23-2a5e-4cd5-ab4c-3f38054041d2 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:55:59.478191 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373d1d23-2a5e-4cd5-ab4c-3f38054041d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:55:59.498711 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-373d1d23-2a5e-4cd5-ab4c-3f38054041d2 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:55:59.499836 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-373d1d23-2a5e-4cd5-ab4c-3f38054041d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:55:59.507810 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1420/2823] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:55:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:55:59.512251 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd97a44-b5bb-4b3b-b3ef-e2c3e2ef99ec None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:55:59.521877 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd97a44-b5bb-4b3b-b3ef-e2c3e2ef99ec None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:55:59.526431 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375f2a74-feb1-48a2-92d2-433806a78f59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:55:59.527035 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375f2a74-feb1-48a2-92d2-433806a78f59 admin admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:55:59.527309 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375f2a74-feb1-48a2-92d2-433806a78f59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:55:59.527727 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375f2a74-feb1-48a2-92d2-433806a78f59 admin admin] PATH_INFO: `/v3/domains` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:55:59.528326 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375f2a74-feb1-48a2-92d2-433806a78f59 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:55:59.528694 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375f2a74-feb1-48a2-92d2-433806a78f59 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:55:59.530325 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375f2a74-feb1-48a2-92d2-433806a78f59 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:55:59.542305 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1404/2824] 10.222.0.45 () {58 vars in 1153 bytes} [Wed Aug 24 17:55:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:55:59.546569 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373d1d23-2a5e-4cd5-ab4c-3f38054041d2 None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:55:59.556380 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373d1d23-2a5e-4cd5-ab4c-3f38054041d2 None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:55:59.561141 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731c86c9-80dd-465a-a2cb-a55445cdcd51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:55:59.561734 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731c86c9-80dd-465a-a2cb-a55445cdcd51 admin admin] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:55:59.562020 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731c86c9-80dd-465a-a2cb-a55445cdcd51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:55:59.562290 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731c86c9-80dd-465a-a2cb-a55445cdcd51 admin admin] PATH_INFO: `/v3/projects` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:55:59.562908 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731c86c9-80dd-465a-a2cb-a55445cdcd51 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:55:59.564837 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731c86c9-80dd-465a-a2cb-a55445cdcd51 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:55:59.592055 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1421/2825] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 17:55:59 2022] POST /identity/v3/projects => generated 367 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:55:59.596122 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375f2a74-feb1-48a2-92d2-433806a78f59 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:55:59.606668 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375f2a74-feb1-48a2-92d2-433806a78f59 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:55:59.661239 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cc04a3-9601-4f85-9bad-5b3475397e15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:55:59.661834 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cc04a3-9601-4f85-9bad-5b3475397e15 admin admin] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:55:59.662114 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cc04a3-9601-4f85-9bad-5b3475397e15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:55:59.662904 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cc04a3-9601-4f85-9bad-5b3475397e15 admin admin] PATH_INFO: `/v3/users` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:55:59.663500 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cc04a3-9601-4f85-9bad-5b3475397e15 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:55:59.665440 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cc04a3-9601-4f85-9bad-5b3475397e15 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:55:59.685825 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1405/2826] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 17:55:59 2022] POST /identity/v3/users => generated 284 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:55:59.690075 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731c86c9-80dd-465a-a2cb-a55445cdcd51 admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:55:59.700455 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731c86c9-80dd-465a-a2cb-a55445cdcd51 admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:55:59.705083 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdad420b-6b1c-4cc2-abe5-d222fd66d585 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:55:59.705681 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdad420b-6b1c-4cc2-abe5-d222fd66d585 admin admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:55:59.705963 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdad420b-6b1c-4cc2-abe5-d222fd66d585 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:55:59.706235 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdad420b-6b1c-4cc2-abe5-d222fd66d585 admin admin] PATH_INFO: `/v3/roles` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:55:59.706877 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdad420b-6b1c-4cc2-abe5-d222fd66d585 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:55:59.708470 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdad420b-6b1c-4cc2-abe5-d222fd66d585 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:55:59.718780 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1422/2827] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:55:59 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:55:59.722641 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cc04a3-9601-4f85-9bad-5b3475397e15 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:55:59.733022 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cc04a3-9601-4f85-9bad-5b3475397e15 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:55:59.737628 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a3353a-b35e-4c21-bc04-a5caf533cdbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:55:59.738218 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a3353a-b35e-4c21-bc04-a5caf533cdbf admin admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:55:59.738496 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a3353a-b35e-4c21-bc04-a5caf533cdbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:55:59.738769 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a3353a-b35e-4c21-bc04-a5caf533cdbf admin admin] PATH_INFO: `/v3/roles` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:55:59.739402 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a3353a-b35e-4c21-bc04-a5caf533cdbf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:55:59.741052 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a3353a-b35e-4c21-bc04-a5caf533cdbf admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:55:59.751331 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1406/2828] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:55:59 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:55:59.755279 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdad420b-6b1c-4cc2-abe5-d222fd66d585 admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:55:59.760639 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=84,keystone:DELETE=13,keystone:INSERT=3 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:55:59.767113 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdad420b-6b1c-4cc2-abe5-d222fd66d585 admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:55:59.774692 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa006742-bc3c-44a2-b22d-09a49e1be099 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:55:59.775358 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa006742-bc3c-44a2-b22d-09a49e1be099 admin admin] REQUEST_METHOD: `PUT` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:55:59.775660 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa006742-bc3c-44a2-b22d-09a49e1be099 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:55:59.775933 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa006742-bc3c-44a2-b22d-09a49e1be099 admin admin] PATH_INFO: `/v3/projects/ee64462e56814da285e38534bef9506a/users/b66a6032019343dfaebd11543c6c9b1b/roles/02476a5b82eb4b188c18a5d706a3ef68` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:55:59.791231 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa006742-bc3c-44a2-b22d-09a49e1be099 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee64462e56814da285e38534bef9506a, user_id=b66a6032019343dfaebd11543c6c9b1b, role_id=02476a5b82eb4b188c18a5d706a3ef68)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:55:59.793907 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa006742-bc3c-44a2-b22d-09a49e1be099 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:55:59.807171 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1423/2829] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 17:55:59 2022] PUT /identity/v3/projects/ee64462e56814da285e38534bef9506a/users/b66a6032019343dfaebd11543c6c9b1b/roles/02476a5b82eb4b188c18a5d706a3ef68 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:55:59.817552 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949208b7-a1a0-480b-98a0-f349743f06c6 None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:55:59.817839 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949208b7-a1a0-480b-98a0-f349743f06c6 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:55:59.818111 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949208b7-a1a0-480b-98a0-f349743f06c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:55:59.845526 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-949208b7-a1a0-480b-98a0-f349743f06c6 None None] MFA Rules not processed for user `b66a6032019343dfaebd11543c6c9b1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:55:59.846745 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-949208b7-a1a0-480b-98a0-f349743f06c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:55:59.864791 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:55:59.910819 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1407/2830] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:55:59 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:55:59.916096 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa006742-bc3c-44a2-b22d-09a49e1be099 admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:55:59.976754 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa006742-bc3c-44a2-b22d-09a49e1be099 admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:55:59.981364 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a98621a-c4fc-4014-be00-ed1e384f6f63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:55:59.981962 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a98621a-c4fc-4014-be00-ed1e384f6f63 admin admin] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:55:59.982244 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a98621a-c4fc-4014-be00-ed1e384f6f63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:55:59.982520 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a98621a-c4fc-4014-be00-ed1e384f6f63 admin admin] PATH_INFO: `/v3/projects` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:55:59.983144 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a98621a-c4fc-4014-be00-ed1e384f6f63 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:55:59.985068 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a98621a-c4fc-4014-be00-ed1e384f6f63 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:00.011093 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1424/2831] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 17:55:59 2022] POST /identity/v3/projects => generated 363 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:56:00.015076 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949208b7-a1a0-480b-98a0-f349743f06c6 None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:00.026135 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949208b7-a1a0-480b-98a0-f349743f06c6 None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:00.080820 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f193be19-0902-45e3-b9a0-35f6ab34a873 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:00.081422 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f193be19-0902-45e3-b9a0-35f6ab34a873 admin admin] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:00.081703 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f193be19-0902-45e3-b9a0-35f6ab34a873 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:00.081978 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f193be19-0902-45e3-b9a0-35f6ab34a873 admin admin] PATH_INFO: `/v3/users` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:00.082599 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f193be19-0902-45e3-b9a0-35f6ab34a873 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:00.085104 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f193be19-0902-45e3-b9a0-35f6ab34a873 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:00.104788 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1408/2832] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 17:56:00 2022] POST /identity/v3/users => generated 288 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:56:00.108836 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a98621a-c4fc-4014-be00-ed1e384f6f63 admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:00.119225 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a98621a-c4fc-4014-be00-ed1e384f6f63 admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:00.123892 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e28c0a-92e9-4f90-9f5d-c450f7ec9653 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:00.124515 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e28c0a-92e9-4f90-9f5d-c450f7ec9653 admin admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:00.124822 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e28c0a-92e9-4f90-9f5d-c450f7ec9653 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:00.125095 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e28c0a-92e9-4f90-9f5d-c450f7ec9653 admin admin] PATH_INFO: `/v3/roles` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:00.125755 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e28c0a-92e9-4f90-9f5d-c450f7ec9653 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:00.127370 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e28c0a-92e9-4f90-9f5d-c450f7ec9653 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:00.138567 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1425/2833] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:56:00 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:56:00.142679 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f193be19-0902-45e3-b9a0-35f6ab34a873 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:00.152989 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f193be19-0902-45e3-b9a0-35f6ab34a873 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:00.157595 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ade03fd-1d15-46f5-a240-75146d124d0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:00.158235 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ade03fd-1d15-46f5-a240-75146d124d0e admin admin] REQUEST_METHOD: `PUT` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:00.158509 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ade03fd-1d15-46f5-a240-75146d124d0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:00.158790 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ade03fd-1d15-46f5-a240-75146d124d0e admin admin] PATH_INFO: `/v3/projects/1e8d832ed8a44542baeb9ff83c59f19d/users/1c800858186f4d4b8211418ca8c7036f/roles/e411ad9eb96e4e8d85b59b5b700ada7b` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:00.173143 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ade03fd-1d15-46f5-a240-75146d124d0e admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e8d832ed8a44542baeb9ff83c59f19d, user_id=1c800858186f4d4b8211418ca8c7036f, role_id=e411ad9eb96e4e8d85b59b5b700ada7b)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:00.175781 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ade03fd-1d15-46f5-a240-75146d124d0e admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:00.188312 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1409/2834] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 17:56:00 2022] PUT /identity/v3/projects/1e8d832ed8a44542baeb9ff83c59f19d/users/1c800858186f4d4b8211418ca8c7036f/roles/e411ad9eb96e4e8d85b59b5b700ada7b => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:56:00.192082 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e28c0a-92e9-4f90-9f5d-c450f7ec9653 admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:00.202648 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e28c0a-92e9-4f90-9f5d-c450f7ec9653 admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:00.256928 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5397d7b0-f1ac-41af-9666-e7e8e73d1d62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:00.257526 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5397d7b0-f1ac-41af-9666-e7e8e73d1d62 admin admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:00.257805 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5397d7b0-f1ac-41af-9666-e7e8e73d1d62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:00.258076 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5397d7b0-f1ac-41af-9666-e7e8e73d1d62 admin admin] PATH_INFO: `/v3/roles` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:00.258708 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5397d7b0-f1ac-41af-9666-e7e8e73d1d62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:00.260578 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5397d7b0-f1ac-41af-9666-e7e8e73d1d62 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:00.270081 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1426/2835] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:56:00 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:56:00.274859 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ade03fd-1d15-46f5-a240-75146d124d0e admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:00.285787 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ade03fd-1d15-46f5-a240-75146d124d0e admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:00.290361 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad9dbaf-ac17-4017-9bd1-50947f280e76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:00.291006 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad9dbaf-ac17-4017-9bd1-50947f280e76 admin admin] REQUEST_METHOD: `PUT` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:00.291285 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad9dbaf-ac17-4017-9bd1-50947f280e76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:00.291559 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad9dbaf-ac17-4017-9bd1-50947f280e76 admin admin] PATH_INFO: `/v3/projects/1e8d832ed8a44542baeb9ff83c59f19d/users/1c800858186f4d4b8211418ca8c7036f/roles/e411ad9eb96e4e8d85b59b5b700ada7b` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:00.293499 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad9dbaf-ac17-4017-9bd1-50947f280e76 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e8d832ed8a44542baeb9ff83c59f19d, user_id=1c800858186f4d4b8211418ca8c7036f, role_id=e411ad9eb96e4e8d85b59b5b700ada7b)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:00.296136 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad9dbaf-ac17-4017-9bd1-50947f280e76 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:00.306375 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1410/2836] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 17:56:00 2022] PUT /identity/v3/projects/1e8d832ed8a44542baeb9ff83c59f19d/users/1c800858186f4d4b8211418ca8c7036f/roles/e411ad9eb96e4e8d85b59b5b700ada7b => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:56:00.317652 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0794bd97-26b5-4943-b78a-fe55af4efb1e None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:00.318149 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0794bd97-26b5-4943-b78a-fe55af4efb1e None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:00.318251 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0794bd97-26b5-4943-b78a-fe55af4efb1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:00.347945 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-0794bd97-26b5-4943-b78a-fe55af4efb1e None None] MFA Rules not processed for user `1c800858186f4d4b8211418ca8c7036f`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:56:00.349169 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-0794bd97-26b5-4943-b78a-fe55af4efb1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:00.385505 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:56:00.416252 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1427/2837] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:56:00 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:56:00.427483 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad9dbaf-ac17-4017-9bd1-50947f280e76 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:00.429824 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-8ad9dbaf-ac17-4017-9bd1-50947f280e76 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:00.486424 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad9dbaf-ac17-4017-9bd1-50947f280e76 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:00.490093 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe4c03c-f686-4836-bf49-35476108e36d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59be342541264942ad87f60ff4de6c8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:00.490714 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe4c03c-f686-4836-bf49-35476108e36d service cinder] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:00.490991 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe4c03c-f686-4836-bf49-35476108e36d service cinder] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:00.491268 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe4c03c-f686-4836-bf49-35476108e36d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:00.500056 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe4c03c-f686-4836-bf49-35476108e36d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:00.501522 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe4c03c-f686-4836-bf49-35476108e36d service cinder] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:00.511206 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1411/2838] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:56:00 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:56:00.590543 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0794bd97-26b5-4943-b78a-fe55af4efb1e None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:00.600722 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0794bd97-26b5-4943-b78a-fe55af4efb1e None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:00.604365 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8355c147-c96a-4dcb-80cc-031cc89332da service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59be342541264942ad87f60ff4de6c8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:00.604999 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8355c147-c96a-4dcb-80cc-031cc89332da service cinder] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:00.605280 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8355c147-c96a-4dcb-80cc-031cc89332da service cinder] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:00.605553 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8355c147-c96a-4dcb-80cc-031cc89332da service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:00.660084 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8355c147-c96a-4dcb-80cc-031cc89332da service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:00.661549 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8355c147-c96a-4dcb-80cc-031cc89332da service cinder] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:00.670772 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1428/2839] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:56:00 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:56:05.326303 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe4c03c-f686-4836-bf49-35476108e36d service cinder] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:05.343771 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:56:05.389239 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe4c03c-f686-4836-bf49-35476108e36d service cinder] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:05.393833 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d8d7f1-ed23-48c4-9ec9-b0f1a8bc03de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:05.394451 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d8d7f1-ed23-48c4-9ec9-b0f1a8bc03de admin admin] REQUEST_METHOD: `DELETE` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:05.394729 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d8d7f1-ed23-48c4-9ec9-b0f1a8bc03de admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:05.394998 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d8d7f1-ed23-48c4-9ec9-b0f1a8bc03de admin admin] PATH_INFO: `/v3/users/b66a6032019343dfaebd11543c6c9b1b` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:05.396119 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d8d7f1-ed23-48c4-9ec9-b0f1a8bc03de admin admin] RBAC: Authorizing `identity:delete_user(user_id=b66a6032019343dfaebd11543c6c9b1b)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:05.398057 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d8d7f1-ed23-48c4-9ec9-b0f1a8bc03de admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:05.442492 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-e1d8d7f1-ed23-48c4-9ec9-b0f1a8bc03de admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b66a6032019343dfaebd11543c6c9b1b'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:05.443064 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-e1d8d7f1-ed23-48c4-9ec9-b0f1a8bc03de admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b66a6032019343dfaebd11543c6c9b1b'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:05.449813 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-e1d8d7f1-ed23-48c4-9ec9-b0f1a8bc03de admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b66a6032019343dfaebd11543c6c9b1b'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:05.450435 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-e1d8d7f1-ed23-48c4-9ec9-b0f1a8bc03de admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b66a6032019343dfaebd11543c6c9b1b'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:05.456776 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-e1d8d7f1-ed23-48c4-9ec9-b0f1a8bc03de admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b66a6032019343dfaebd11543c6c9b1b'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:05.473711 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1412/2840] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 17:56:05 2022] DELETE /identity/v3/users/b66a6032019343dfaebd11543c6c9b1b => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:56:05.478634 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea453f9-3ed2-4e2e-8e8e-78a275c7baf8 None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:05.479125 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea453f9-3ed2-4e2e-8e8e-78a275c7baf8 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:05.479254 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea453f9-3ed2-4e2e-8e8e-78a275c7baf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:05.497959 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-fea453f9-3ed2-4e2e-8e8e-78a275c7baf8 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:56:05.499134 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-fea453f9-3ed2-4e2e-8e8e-78a275c7baf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:05.555989 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1429/2841] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:56:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:56:05.565629 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d8d7f1-ed23-48c4-9ec9-b0f1a8bc03de admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:05.566994 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-e1d8d7f1-ed23-48c4-9ec9-b0f1a8bc03de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:05.623718 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d8d7f1-ed23-48c4-9ec9-b0f1a8bc03de admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:05.627422 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8192e8cf-4e37-45e3-b3a5-8d1a43e5ea65 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43ad98a250a144099ccd18378aedd4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:05.628022 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8192e8cf-4e37-45e3-b3a5-8d1a43e5ea65 service neutron] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:05.628318 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8192e8cf-4e37-45e3-b3a5-8d1a43e5ea65 service neutron] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:05.628595 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8192e8cf-4e37-45e3-b3a5-8d1a43e5ea65 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:05.637628 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8192e8cf-4e37-45e3-b3a5-8d1a43e5ea65 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:05.639050 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8192e8cf-4e37-45e3-b3a5-8d1a43e5ea65 service neutron] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:05.648201 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1413/2842] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:56:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:56:05.907141 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea453f9-3ed2-4e2e-8e8e-78a275c7baf8 None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:05.908610 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-fea453f9-3ed2-4e2e-8e8e-78a275c7baf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:05.927159 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea453f9-3ed2-4e2e-8e8e-78a275c7baf8 None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:05.931883 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504c54bd-c43a-421c-8c13-0583c4d480e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:05.932524 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504c54bd-c43a-421c-8c13-0583c4d480e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:05.932909 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504c54bd-c43a-421c-8c13-0583c4d480e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:05.933084 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504c54bd-c43a-421c-8c13-0583c4d480e3 admin admin] PATH_INFO: `/v3/projects/ee64462e56814da285e38534bef9506a` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:05.934284 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504c54bd-c43a-421c-8c13-0583c4d480e3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ee64462e56814da285e38534bef9506a)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:05.936120 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504c54bd-c43a-421c-8c13-0583c4d480e3 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:05.981964 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-504c54bd-c43a-421c-8c13-0583c4d480e3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ee64462e56814da285e38534bef9506a', 'request_id': 'req-504c54bd-c43a-421c-8c13-0583c4d480e3'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:05.989106 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-504c54bd-c43a-421c-8c13-0583c4d480e3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ee64462e56814da285e38534bef9506a', 'request_id': 'req-504c54bd-c43a-421c-8c13-0583c4d480e3'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:05.995385 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-504c54bd-c43a-421c-8c13-0583c4d480e3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ee64462e56814da285e38534bef9506a', 'request_id': 'req-504c54bd-c43a-421c-8c13-0583c4d480e3'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:06.000000 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:56:06.006967 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-504c54bd-c43a-421c-8c13-0583c4d480e3 admin admin] The token cache is being invalidate because project ee64462e56814da285e38534bef9506a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81758) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:56:06.007480 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-504c54bd-c43a-421c-8c13-0583c4d480e3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:06.008500 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-504c54bd-c43a-421c-8c13-0583c4d480e3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:06.010544 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1430/2843] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 17:56:05 2022] DELETE /identity/v3/projects/ee64462e56814da285e38534bef9506a => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:56:06.014539 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8192e8cf-4e37-45e3-b3a5-8d1a43e5ea65 service neutron] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:06.024484 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8192e8cf-4e37-45e3-b3a5-8d1a43e5ea65 service neutron] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:06.057597 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:56:06.081853 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beeb6f0f-aef3-4f8b-97f6-b1908026a7ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:06.082485 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beeb6f0f-aef3-4f8b-97f6-b1908026a7ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:06.082764 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beeb6f0f-aef3-4f8b-97f6-b1908026a7ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:06.083038 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beeb6f0f-aef3-4f8b-97f6-b1908026a7ed admin admin] PATH_INFO: `/v3/users/1c800858186f4d4b8211418ca8c7036f` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:06.084752 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beeb6f0f-aef3-4f8b-97f6-b1908026a7ed admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c800858186f4d4b8211418ca8c7036f)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:06.086685 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beeb6f0f-aef3-4f8b-97f6-b1908026a7ed admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:06.129869 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-beeb6f0f-aef3-4f8b-97f6-b1908026a7ed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c800858186f4d4b8211418ca8c7036f'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:06.131063 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-beeb6f0f-aef3-4f8b-97f6-b1908026a7ed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c800858186f4d4b8211418ca8c7036f'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:06.138253 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-beeb6f0f-aef3-4f8b-97f6-b1908026a7ed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c800858186f4d4b8211418ca8c7036f'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:06.138905 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-beeb6f0f-aef3-4f8b-97f6-b1908026a7ed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c800858186f4d4b8211418ca8c7036f'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:06.145759 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-beeb6f0f-aef3-4f8b-97f6-b1908026a7ed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c800858186f4d4b8211418ca8c7036f'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:06.164158 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1414/2844] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 17:56:06 2022] DELETE /identity/v3/users/1c800858186f4d4b8211418ca8c7036f => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:56:06.429948 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504c54bd-c43a-421c-8c13-0583c4d480e3 admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:06.431448 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-504c54bd-c43a-421c-8c13-0583c4d480e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:06.445344 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504c54bd-c43a-421c-8c13-0583c4d480e3 admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:06.501677 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714c945f-6c7f-414a-8d88-ca7ae33a3079 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:06.502314 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714c945f-6c7f-414a-8d88-ca7ae33a3079 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:06.502645 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714c945f-6c7f-414a-8d88-ca7ae33a3079 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:06.502884 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714c945f-6c7f-414a-8d88-ca7ae33a3079 admin admin] PATH_INFO: `/v3/projects/1e8d832ed8a44542baeb9ff83c59f19d` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:06.503988 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714c945f-6c7f-414a-8d88-ca7ae33a3079 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e8d832ed8a44542baeb9ff83c59f19d)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:06.506617 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714c945f-6c7f-414a-8d88-ca7ae33a3079 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:06.551516 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-714c945f-6c7f-414a-8d88-ca7ae33a3079 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e8d832ed8a44542baeb9ff83c59f19d', 'request_id': 'req-714c945f-6c7f-414a-8d88-ca7ae33a3079'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:06.559705 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-714c945f-6c7f-414a-8d88-ca7ae33a3079 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e8d832ed8a44542baeb9ff83c59f19d', 'request_id': 'req-714c945f-6c7f-414a-8d88-ca7ae33a3079'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:06.565806 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-714c945f-6c7f-414a-8d88-ca7ae33a3079 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e8d832ed8a44542baeb9ff83c59f19d', 'request_id': 'req-714c945f-6c7f-414a-8d88-ca7ae33a3079'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:06.576850 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-714c945f-6c7f-414a-8d88-ca7ae33a3079 admin admin] The token cache is being invalidate because project 1e8d832ed8a44542baeb9ff83c59f19d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81758) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:56:06.577140 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-714c945f-6c7f-414a-8d88-ca7ae33a3079 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:06.577771 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-714c945f-6c7f-414a-8d88-ca7ae33a3079 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:06.579260 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1431/2845] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 17:56:06 2022] DELETE /identity/v3/projects/1e8d832ed8a44542baeb9ff83c59f19d => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:56:06.584219 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28a63a4-d238-43f0-b69d-bda00878d652 None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:06.584728 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28a63a4-d238-43f0-b69d-bda00878d652 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:06.584845 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28a63a4-d238-43f0-b69d-bda00878d652 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:06.603751 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-d28a63a4-d238-43f0-b69d-bda00878d652 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:56:06.605006 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-d28a63a4-d238-43f0-b69d-bda00878d652 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:06.663001 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1415/2846] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:56:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:56:06.670736 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e4ca66-f4c9-4c7d-8ef9-26e4d1622c2d None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:06.671050 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e4ca66-f4c9-4c7d-8ef9-26e4d1622c2d None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:06.671323 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e4ca66-f4c9-4c7d-8ef9-26e4d1622c2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:06.690772 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-74e4ca66-f4c9-4c7d-8ef9-26e4d1622c2d None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:56:06.691964 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-74e4ca66-f4c9-4c7d-8ef9-26e4d1622c2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:06.700452 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1432/2847] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:56:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:56:06.704654 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28a63a4-d238-43f0-b69d-bda00878d652 None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:06.714294 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28a63a4-d238-43f0-b69d-bda00878d652 None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:06.718909 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de57e50-b79d-4c63-a7ab-6087c590bd6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:06.719503 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de57e50-b79d-4c63-a7ab-6087c590bd6a admin admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:06.719782 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de57e50-b79d-4c63-a7ab-6087c590bd6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:06.720057 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de57e50-b79d-4c63-a7ab-6087c590bd6a admin admin] PATH_INFO: `/v3/domains` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:06.720733 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de57e50-b79d-4c63-a7ab-6087c590bd6a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81759) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:56:06.721037 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de57e50-b79d-4c63-a7ab-6087c590bd6a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:06.722631 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de57e50-b79d-4c63-a7ab-6087c590bd6a admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:06.733852 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1416/2848] 10.222.0.45 () {58 vars in 1153 bytes} [Wed Aug 24 17:56:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:56:06.737945 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e4ca66-f4c9-4c7d-8ef9-26e4d1622c2d None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:06.747505 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e4ca66-f4c9-4c7d-8ef9-26e4d1622c2d None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:06.752241 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fefba04-b14c-4ab9-a762-1faa1230955c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:06.752867 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fefba04-b14c-4ab9-a762-1faa1230955c admin admin] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:06.753147 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fefba04-b14c-4ab9-a762-1faa1230955c admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:06.753415 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fefba04-b14c-4ab9-a762-1faa1230955c admin admin] PATH_INFO: `/v3/projects` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:06.754040 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fefba04-b14c-4ab9-a762-1faa1230955c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:06.756577 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fefba04-b14c-4ab9-a762-1faa1230955c admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:06.783239 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1433/2849] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 17:56:06 2022] POST /identity/v3/projects => generated 385 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:56:06.787377 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de57e50-b79d-4c63-a7ab-6087c590bd6a admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:06.797412 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de57e50-b79d-4c63-a7ab-6087c590bd6a admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:06.840475 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:56:06.856456 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff36d44-bf0d-4fcb-a7a3-5cac4e415d07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:06.857103 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff36d44-bf0d-4fcb-a7a3-5cac4e415d07 admin admin] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:06.857384 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff36d44-bf0d-4fcb-a7a3-5cac4e415d07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:06.857659 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff36d44-bf0d-4fcb-a7a3-5cac4e415d07 admin admin] PATH_INFO: `/v3/users` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:06.858274 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff36d44-bf0d-4fcb-a7a3-5cac4e415d07 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:06.860182 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff36d44-bf0d-4fcb-a7a3-5cac4e415d07 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:06.880745 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1417/2850] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 17:56:06 2022] POST /identity/v3/users => generated 293 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:56:06.885108 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fefba04-b14c-4ab9-a762-1faa1230955c admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:06.895117 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fefba04-b14c-4ab9-a762-1faa1230955c admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:06.899667 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5224cb-90cd-4cf8-84a3-73fd442d53c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:06.900258 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5224cb-90cd-4cf8-84a3-73fd442d53c3 admin admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:06.900535 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5224cb-90cd-4cf8-84a3-73fd442d53c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:06.900835 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5224cb-90cd-4cf8-84a3-73fd442d53c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:06.901468 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5224cb-90cd-4cf8-84a3-73fd442d53c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:06.903048 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5224cb-90cd-4cf8-84a3-73fd442d53c3 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:06.913624 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1434/2851] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:56:06 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:56:06.917563 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff36d44-bf0d-4fcb-a7a3-5cac4e415d07 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:06.928075 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff36d44-bf0d-4fcb-a7a3-5cac4e415d07 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:06.932724 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbeab364-35f0-4c90-9429-e05c8bfbd2dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:06.933313 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeab364-35f0-4c90-9429-e05c8bfbd2dc admin admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:06.933594 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeab364-35f0-4c90-9429-e05c8bfbd2dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:06.933873 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeab364-35f0-4c90-9429-e05c8bfbd2dc admin admin] PATH_INFO: `/v3/roles` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:06.934512 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbeab364-35f0-4c90-9429-e05c8bfbd2dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:06.936094 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbeab364-35f0-4c90-9429-e05c8bfbd2dc admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:06.946374 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1418/2852] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:56:06 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 17:56:06.950234 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5224cb-90cd-4cf8-84a3-73fd442d53c3 admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:06.959920 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5224cb-90cd-4cf8-84a3-73fd442d53c3 admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:06.964522 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e9aad6-9356-4df4-a844-037a30b1d780 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:06.965188 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e9aad6-9356-4df4-a844-037a30b1d780 admin admin] REQUEST_METHOD: `PUT` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:06.965469 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e9aad6-9356-4df4-a844-037a30b1d780 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:06.965744 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e9aad6-9356-4df4-a844-037a30b1d780 admin admin] PATH_INFO: `/v3/projects/283b1e94325242c4bc2e2ba329032356/users/e3005b1b3cf44426aeab606b1b3d8649/roles/02476a5b82eb4b188c18a5d706a3ef68` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:06.980235 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e9aad6-9356-4df4-a844-037a30b1d780 admin admin] RBAC: Authorizing `identity:create_grant(project_id=283b1e94325242c4bc2e2ba329032356, user_id=e3005b1b3cf44426aeab606b1b3d8649, role_id=02476a5b82eb4b188c18a5d706a3ef68)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:06.982913 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e9aad6-9356-4df4-a844-037a30b1d780 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:06.995895 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1435/2853] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 17:56:06 2022] PUT /identity/v3/projects/283b1e94325242c4bc2e2ba329032356/users/e3005b1b3cf44426aeab606b1b3d8649/roles/02476a5b82eb4b188c18a5d706a3ef68 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:56:07.006418 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f8ead8-94d9-4585-92bd-3ee4d11f57e0 None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:07.006850 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f8ead8-94d9-4585-92bd-3ee4d11f57e0 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:07.007127 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f8ead8-94d9-4585-92bd-3ee4d11f57e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:07.033839 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-54f8ead8-94d9-4585-92bd-3ee4d11f57e0 None None] MFA Rules not processed for user `e3005b1b3cf44426aeab606b1b3d8649`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:56:07.035030 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-54f8ead8-94d9-4585-92bd-3ee4d11f57e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:07.096978 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1419/2854] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:56:07 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:56:07.101808 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e9aad6-9356-4df4-a844-037a30b1d780 admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:07.126634 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:56:07.164218 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e9aad6-9356-4df4-a844-037a30b1d780 admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:07.168841 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81eb3450-fdfb-4db2-a143-f550643d2860 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:07.169438 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81eb3450-fdfb-4db2-a143-f550643d2860 admin admin] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:07.169720 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81eb3450-fdfb-4db2-a143-f550643d2860 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:07.169989 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81eb3450-fdfb-4db2-a143-f550643d2860 admin admin] PATH_INFO: `/v3/projects` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:07.170601 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81eb3450-fdfb-4db2-a143-f550643d2860 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:07.172493 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81eb3450-fdfb-4db2-a143-f550643d2860 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:07.198269 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1436/2855] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 17:56:07 2022] POST /identity/v3/projects => generated 385 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:56:07.202318 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f8ead8-94d9-4585-92bd-3ee4d11f57e0 None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:07.213572 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f8ead8-94d9-4585-92bd-3ee4d11f57e0 None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:07.267991 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a9107f-996f-4818-86f0-6ea0b60e2942 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:07.268612 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a9107f-996f-4818-86f0-6ea0b60e2942 admin admin] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:07.268896 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a9107f-996f-4818-86f0-6ea0b60e2942 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:07.269171 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a9107f-996f-4818-86f0-6ea0b60e2942 admin admin] PATH_INFO: `/v3/users` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:07.269811 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a9107f-996f-4818-86f0-6ea0b60e2942 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:07.272310 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a9107f-996f-4818-86f0-6ea0b60e2942 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:07.292708 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1420/2856] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 17:56:07 2022] POST /identity/v3/users => generated 293 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:56:07.296376 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81eb3450-fdfb-4db2-a143-f550643d2860 admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:07.307366 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81eb3450-fdfb-4db2-a143-f550643d2860 admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:07.311984 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554e2453-a8f3-4cb5-a376-66cc4f97fddf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:07.312603 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554e2453-a8f3-4cb5-a376-66cc4f97fddf admin admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:07.312886 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554e2453-a8f3-4cb5-a376-66cc4f97fddf admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:07.313161 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554e2453-a8f3-4cb5-a376-66cc4f97fddf admin admin] PATH_INFO: `/v3/roles` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:07.313802 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554e2453-a8f3-4cb5-a376-66cc4f97fddf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:07.315492 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554e2453-a8f3-4cb5-a376-66cc4f97fddf admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:07.326548 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1437/2857] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:56:07 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:56:07.331077 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a9107f-996f-4818-86f0-6ea0b60e2942 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:07.341081 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a9107f-996f-4818-86f0-6ea0b60e2942 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:07.345658 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6186cf0-a580-48f7-a55c-946d65154f72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:07.346248 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6186cf0-a580-48f7-a55c-946d65154f72 admin admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:07.346524 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6186cf0-a580-48f7-a55c-946d65154f72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:07.346797 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6186cf0-a580-48f7-a55c-946d65154f72 admin admin] PATH_INFO: `/v3/roles` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:07.347482 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6186cf0-a580-48f7-a55c-946d65154f72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:07.349055 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6186cf0-a580-48f7-a55c-946d65154f72 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:07.358798 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1421/2858] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:56:07 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:56:07.362739 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554e2453-a8f3-4cb5-a376-66cc4f97fddf admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:07.372846 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554e2453-a8f3-4cb5-a376-66cc4f97fddf admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:07.377338 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d363ad9-b025-44ed-a4d5-f6179f2b881f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:07.377968 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d363ad9-b025-44ed-a4d5-f6179f2b881f admin admin] REQUEST_METHOD: `PUT` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:07.378244 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d363ad9-b025-44ed-a4d5-f6179f2b881f admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:07.378510 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d363ad9-b025-44ed-a4d5-f6179f2b881f admin admin] PATH_INFO: `/v3/projects/4351320fd5184ac385fc403376bc9778/users/32b1a8067ca14593a3490ab5ddf500f5/roles/02476a5b82eb4b188c18a5d706a3ef68` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:07.393230 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d363ad9-b025-44ed-a4d5-f6179f2b881f admin admin] RBAC: Authorizing `identity:create_grant(project_id=4351320fd5184ac385fc403376bc9778, user_id=32b1a8067ca14593a3490ab5ddf500f5, role_id=02476a5b82eb4b188c18a5d706a3ef68)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:07.395906 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d363ad9-b025-44ed-a4d5-f6179f2b881f admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:07.410435 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1438/2859] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 17:56:07 2022] PUT /identity/v3/projects/4351320fd5184ac385fc403376bc9778/users/32b1a8067ca14593a3490ab5ddf500f5/roles/02476a5b82eb4b188c18a5d706a3ef68 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:56:07.421537 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80136df0-9594-4930-8909-c16a9cbe3c39 None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:07.421887 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80136df0-9594-4930-8909-c16a9cbe3c39 None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:07.422145 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80136df0-9594-4930-8909-c16a9cbe3c39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:07.449833 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-80136df0-9594-4930-8909-c16a9cbe3c39 None None] MFA Rules not processed for user `32b1a8067ca14593a3490ab5ddf500f5`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:56:07.451020 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-80136df0-9594-4930-8909-c16a9cbe3c39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:07.468396 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:56:07.515175 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1422/2860] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:56:07 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:56:07.519523 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d363ad9-b025-44ed-a4d5-f6179f2b881f admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:07.578958 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d363ad9-b025-44ed-a4d5-f6179f2b881f admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:07.583539 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552414e2-abce-4a70-8f57-e7f8906f8831 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:07.584130 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552414e2-abce-4a70-8f57-e7f8906f8831 admin admin] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:07.584405 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552414e2-abce-4a70-8f57-e7f8906f8831 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:07.584706 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552414e2-abce-4a70-8f57-e7f8906f8831 admin admin] PATH_INFO: `/v3/projects` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:07.585326 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552414e2-abce-4a70-8f57-e7f8906f8831 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:07.587200 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552414e2-abce-4a70-8f57-e7f8906f8831 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:07.613424 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1439/2861] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 17:56:07 2022] POST /identity/v3/projects => generated 381 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:56:07.617938 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80136df0-9594-4930-8909-c16a9cbe3c39 None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:07.628361 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80136df0-9594-4930-8909-c16a9cbe3c39 None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:07.682932 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2048292e-b396-4f07-a29b-e52f915bc9d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:07.683545 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2048292e-b396-4f07-a29b-e52f915bc9d7 admin admin] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:07.683829 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2048292e-b396-4f07-a29b-e52f915bc9d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:07.684106 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2048292e-b396-4f07-a29b-e52f915bc9d7 admin admin] PATH_INFO: `/v3/users` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:07.684765 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2048292e-b396-4f07-a29b-e52f915bc9d7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:07.686736 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2048292e-b396-4f07-a29b-e52f915bc9d7 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:07.709828 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1423/2862] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 17:56:07 2022] POST /identity/v3/users => generated 297 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:56:07.710970 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552414e2-abce-4a70-8f57-e7f8906f8831 admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:07.721068 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552414e2-abce-4a70-8f57-e7f8906f8831 admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:07.726405 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f6bb6c-3851-445d-91de-cb08313ca936 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:07.727009 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f6bb6c-3851-445d-91de-cb08313ca936 admin admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:07.727320 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f6bb6c-3851-445d-91de-cb08313ca936 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:07.727593 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f6bb6c-3851-445d-91de-cb08313ca936 admin admin] PATH_INFO: `/v3/roles` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:07.728235 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f6bb6c-3851-445d-91de-cb08313ca936 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:07.729904 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f6bb6c-3851-445d-91de-cb08313ca936 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:07.740968 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1440/2863] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:56:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 17:56:07.744879 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2048292e-b396-4f07-a29b-e52f915bc9d7 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:07.754602 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2048292e-b396-4f07-a29b-e52f915bc9d7 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:07.759237 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8443e83f-4597-41bf-8a94-0810ce97ce63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:07.760137 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8443e83f-4597-41bf-8a94-0810ce97ce63 admin admin] REQUEST_METHOD: `PUT` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:07.760418 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8443e83f-4597-41bf-8a94-0810ce97ce63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:07.760734 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8443e83f-4597-41bf-8a94-0810ce97ce63 admin admin] PATH_INFO: `/v3/projects/dd2a8882876b46ac85edeab6cd0b8805/users/03766f29c1ef462f8d206c20ad26c8b5/roles/e411ad9eb96e4e8d85b59b5b700ada7b` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:07.776885 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8443e83f-4597-41bf-8a94-0810ce97ce63 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd2a8882876b46ac85edeab6cd0b8805, user_id=03766f29c1ef462f8d206c20ad26c8b5, role_id=e411ad9eb96e4e8d85b59b5b700ada7b)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:07.779579 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8443e83f-4597-41bf-8a94-0810ce97ce63 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:07.793600 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1424/2864] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 17:56:07 2022] PUT /identity/v3/projects/dd2a8882876b46ac85edeab6cd0b8805/users/03766f29c1ef462f8d206c20ad26c8b5/roles/e411ad9eb96e4e8d85b59b5b700ada7b => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:56:07.797566 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f6bb6c-3851-445d-91de-cb08313ca936 admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:07.808210 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f6bb6c-3851-445d-91de-cb08313ca936 admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:07.850006 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:56:07.862878 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48255f18-26cc-4cf2-86d0-de751b59c508 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:07.863529 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48255f18-26cc-4cf2-86d0-de751b59c508 admin admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:07.863757 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48255f18-26cc-4cf2-86d0-de751b59c508 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:07.864029 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48255f18-26cc-4cf2-86d0-de751b59c508 admin admin] PATH_INFO: `/v3/roles` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:07.864704 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48255f18-26cc-4cf2-86d0-de751b59c508 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:07.866863 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48255f18-26cc-4cf2-86d0-de751b59c508 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:07.876285 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1441/2865] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:56:07 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:56:07.880430 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8443e83f-4597-41bf-8a94-0810ce97ce63 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:07.890794 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8443e83f-4597-41bf-8a94-0810ce97ce63 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:07.895326 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287d200a-0873-42f7-9d21-b976da81bc92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:07.895985 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287d200a-0873-42f7-9d21-b976da81bc92 admin admin] REQUEST_METHOD: `PUT` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:07.896269 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287d200a-0873-42f7-9d21-b976da81bc92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:07.896543 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287d200a-0873-42f7-9d21-b976da81bc92 admin admin] PATH_INFO: `/v3/projects/dd2a8882876b46ac85edeab6cd0b8805/users/03766f29c1ef462f8d206c20ad26c8b5/roles/e411ad9eb96e4e8d85b59b5b700ada7b` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:07.898466 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287d200a-0873-42f7-9d21-b976da81bc92 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd2a8882876b46ac85edeab6cd0b8805, user_id=03766f29c1ef462f8d206c20ad26c8b5, role_id=e411ad9eb96e4e8d85b59b5b700ada7b)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:07.901155 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287d200a-0873-42f7-9d21-b976da81bc92 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:07.911214 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1425/2866] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 17:56:07 2022] PUT /identity/v3/projects/dd2a8882876b46ac85edeab6cd0b8805/users/03766f29c1ef462f8d206c20ad26c8b5/roles/e411ad9eb96e4e8d85b59b5b700ada7b => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:56:07.922222 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0a2b56-81e4-4887-a315-01ec4725f32e None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:07.922532 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0a2b56-81e4-4887-a315-01ec4725f32e None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:07.922805 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0a2b56-81e4-4887-a315-01ec4725f32e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:07.950184 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-cc0a2b56-81e4-4887-a315-01ec4725f32e None None] MFA Rules not processed for user `03766f29c1ef462f8d206c20ad26c8b5`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:56:07.951373 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-cc0a2b56-81e4-4887-a315-01ec4725f32e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:08.016677 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1442/2867] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:56:07 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:56:08.028273 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287d200a-0873-42f7-9d21-b976da81bc92 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:08.030004 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-287d200a-0873-42f7-9d21-b976da81bc92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:08.086513 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287d200a-0873-42f7-9d21-b976da81bc92 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:08.090121 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032e9c1e-1f54-4ac6-b417-f69ca92ef86a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59be342541264942ad87f60ff4de6c8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:08.090719 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032e9c1e-1f54-4ac6-b417-f69ca92ef86a service cinder] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:08.091233 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032e9c1e-1f54-4ac6-b417-f69ca92ef86a service cinder] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:08.091515 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032e9c1e-1f54-4ac6-b417-f69ca92ef86a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:08.116874 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:56:08.147242 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032e9c1e-1f54-4ac6-b417-f69ca92ef86a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:08.148708 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032e9c1e-1f54-4ac6-b417-f69ca92ef86a service cinder] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:08.157427 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1426/2868] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:56:08 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:56:09.642770 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0a2b56-81e4-4887-a315-01ec4725f32e None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:09.644199 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-cc0a2b56-81e4-4887-a315-01ec4725f32e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:09.656328 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0a2b56-81e4-4887-a315-01ec4725f32e None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:09.659802 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633c4c07-d3e3-4a0b-9650-dea15cb29056 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59be342541264942ad87f60ff4de6c8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:09.660392 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633c4c07-d3e3-4a0b-9650-dea15cb29056 service cinder] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:09.660737 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633c4c07-d3e3-4a0b-9650-dea15cb29056 service cinder] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:09.661018 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633c4c07-d3e3-4a0b-9650-dea15cb29056 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:09.717658 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633c4c07-d3e3-4a0b-9650-dea15cb29056 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:09.719007 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633c4c07-d3e3-4a0b-9650-dea15cb29056 service cinder] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:09.728319 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1443/2869] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:56:09 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:56:10.240696 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032e9c1e-1f54-4ac6-b417-f69ca92ef86a service cinder] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:10.249295 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032e9c1e-1f54-4ac6-b417-f69ca92ef86a service cinder] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:10.252894 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d828912-289a-4cfe-a808-416ffd50c019 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59be342541264942ad87f60ff4de6c8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:10.253496 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d828912-289a-4cfe-a808-416ffd50c019 service cinder] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:10.253776 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d828912-289a-4cfe-a808-416ffd50c019 service cinder] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:10.254050 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d828912-289a-4cfe-a808-416ffd50c019 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:10.263753 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d828912-289a-4cfe-a808-416ffd50c019 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:10.265267 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d828912-289a-4cfe-a808-416ffd50c019 service cinder] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:10.274648 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1427/2870] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:56:10 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:56:14.121192 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633c4c07-d3e3-4a0b-9650-dea15cb29056 service cinder] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:14.183499 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633c4c07-d3e3-4a0b-9650-dea15cb29056 service cinder] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:14.188172 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9e9cb5-db98-43e3-9f07-6b2b8db96f81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:14.188834 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9e9cb5-db98-43e3-9f07-6b2b8db96f81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:14.189117 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9e9cb5-db98-43e3-9f07-6b2b8db96f81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:14.189393 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9e9cb5-db98-43e3-9f07-6b2b8db96f81 admin admin] PATH_INFO: `/v3/users/e3005b1b3cf44426aeab606b1b3d8649` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:14.190511 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9e9cb5-db98-43e3-9f07-6b2b8db96f81 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e3005b1b3cf44426aeab606b1b3d8649)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:14.192434 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9e9cb5-db98-43e3-9f07-6b2b8db96f81 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:14.197709 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=100 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:56:14.241204 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-8f9e9cb5-db98-43e3-9f07-6b2b8db96f81 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e3005b1b3cf44426aeab606b1b3d8649'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:14.241861 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-8f9e9cb5-db98-43e3-9f07-6b2b8db96f81 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e3005b1b3cf44426aeab606b1b3d8649'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:14.249155 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-8f9e9cb5-db98-43e3-9f07-6b2b8db96f81 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e3005b1b3cf44426aeab606b1b3d8649'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:14.249790 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-8f9e9cb5-db98-43e3-9f07-6b2b8db96f81 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e3005b1b3cf44426aeab606b1b3d8649'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:14.262177 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-8f9e9cb5-db98-43e3-9f07-6b2b8db96f81 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e3005b1b3cf44426aeab606b1b3d8649'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:14.276450 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1444/2871] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 17:56:14 2022] DELETE /identity/v3/users/e3005b1b3cf44426aeab606b1b3d8649 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:56:14.280862 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a31b78-7e18-4425-b0a6-6400e1e0daea None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:14.281370 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a31b78-7e18-4425-b0a6-6400e1e0daea None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:14.281475 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a31b78-7e18-4425-b0a6-6400e1e0daea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:14.284524 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9e9cb5-db98-43e3-9f07-6b2b8db96f81 admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:14.285936 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-8f9e9cb5-db98-43e3-9f07-6b2b8db96f81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:14.300436 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-63a31b78-7e18-4425-b0a6-6400e1e0daea None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:56:14.301655 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-63a31b78-7e18-4425-b0a6-6400e1e0daea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:14.345411 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9e9cb5-db98-43e3-9f07-6b2b8db96f81 admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:14.349091 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd0c96e-1db1-4ff3-80d7-22e2a51deddf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43ad98a250a144099ccd18378aedd4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:14.349705 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd0c96e-1db1-4ff3-80d7-22e2a51deddf service neutron] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:14.350149 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd0c96e-1db1-4ff3-80d7-22e2a51deddf service neutron] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:14.350432 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd0c96e-1db1-4ff3-80d7-22e2a51deddf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:14.352284 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-abd0c96e-1db1-4ff3-80d7-22e2a51deddf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:14.360090 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1428/2872] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:56:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:56:14.369687 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a31b78-7e18-4425-b0a6-6400e1e0daea None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:14.371010 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-63a31b78-7e18-4425-b0a6-6400e1e0daea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:14.383904 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a31b78-7e18-4425-b0a6-6400e1e0daea None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:14.387864 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46387f56-4d22-458b-b072-d01b8a8989cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43ad98a250a144099ccd18378aedd4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:14.388459 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46387f56-4d22-458b-b072-d01b8a8989cb service neutron] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:14.388758 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46387f56-4d22-458b-b072-d01b8a8989cb service neutron] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:14.389031 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46387f56-4d22-458b-b072-d01b8a8989cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:14.398352 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46387f56-4d22-458b-b072-d01b8a8989cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:14.399767 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46387f56-4d22-458b-b072-d01b8a8989cb service neutron] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:14.409107 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1429/2873] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:56:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 17:56:14.423402 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd0c96e-1db1-4ff3-80d7-22e2a51deddf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:14.424889 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd0c96e-1db1-4ff3-80d7-22e2a51deddf service neutron] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:14.435937 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1445/2874] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:56:14 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 17:56:14.814923 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46387f56-4d22-458b-b072-d01b8a8989cb service neutron] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:14.816374 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-46387f56-4d22-458b-b072-d01b8a8989cb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:14.829616 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46387f56-4d22-458b-b072-d01b8a8989cb service neutron] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:14.834275 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0f3cd6-0fc0-4a19-a519-9d9b190e9304 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:14.834950 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0f3cd6-0fc0-4a19-a519-9d9b190e9304 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:14.835238 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0f3cd6-0fc0-4a19-a519-9d9b190e9304 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:14.835511 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0f3cd6-0fc0-4a19-a519-9d9b190e9304 admin admin] PATH_INFO: `/v3/projects/283b1e94325242c4bc2e2ba329032356` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:14.836702 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0f3cd6-0fc0-4a19-a519-9d9b190e9304 admin admin] RBAC: Authorizing `identity:delete_project(project_id=283b1e94325242c4bc2e2ba329032356)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:14.838758 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0f3cd6-0fc0-4a19-a519-9d9b190e9304 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:14.888412 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-ec0f3cd6-0fc0-4a19-a519-9d9b190e9304 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '283b1e94325242c4bc2e2ba329032356', 'request_id': 'req-ec0f3cd6-0fc0-4a19-a519-9d9b190e9304'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:14.894261 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-ec0f3cd6-0fc0-4a19-a519-9d9b190e9304 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '283b1e94325242c4bc2e2ba329032356', 'request_id': 'req-ec0f3cd6-0fc0-4a19-a519-9d9b190e9304'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:14.900631 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-ec0f3cd6-0fc0-4a19-a519-9d9b190e9304 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '283b1e94325242c4bc2e2ba329032356', 'request_id': 'req-ec0f3cd6-0fc0-4a19-a519-9d9b190e9304'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:14.916153 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-ec0f3cd6-0fc0-4a19-a519-9d9b190e9304 admin admin] The token cache is being invalidate because project 283b1e94325242c4bc2e2ba329032356 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81759) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:56:14.916374 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-ec0f3cd6-0fc0-4a19-a519-9d9b190e9304 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:14.917023 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-ec0f3cd6-0fc0-4a19-a519-9d9b190e9304 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:14.918737 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1430/2875] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 17:56:14 2022] DELETE /identity/v3/projects/283b1e94325242c4bc2e2ba329032356 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:56:14.922585 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd0c96e-1db1-4ff3-80d7-22e2a51deddf service neutron] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:14.933570 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd0c96e-1db1-4ff3-80d7-22e2a51deddf service neutron] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:14.974530 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0f3cd6-0fc0-4a19-a519-9d9b190e9304 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:14.976017 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-ec0f3cd6-0fc0-4a19-a519-9d9b190e9304 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:15.021419 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:56:15.026918 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:56:15.080946 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564af720-daa7-46d1-a602-d9658c6c07e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:15.081526 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0f3cd6-0fc0-4a19-a519-9d9b190e9304 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:15.081669 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564af720-daa7-46d1-a602-d9658c6c07e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:15.081864 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564af720-daa7-46d1-a602-d9658c6c07e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:15.082140 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564af720-daa7-46d1-a602-d9658c6c07e2 admin admin] PATH_INFO: `/v3/users/32b1a8067ca14593a3490ab5ddf500f5` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:15.083410 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564af720-daa7-46d1-a602-d9658c6c07e2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=32b1a8067ca14593a3490ab5ddf500f5)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:15.085598 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564af720-daa7-46d1-a602-d9658c6c07e2 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:15.089638 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bef4864-880d-4f1b-bce7-792deec6e8c7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '053b7ce6d6f649ed91a26e434dab805e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:15.090262 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bef4864-880d-4f1b-bce7-792deec6e8c7 service nova] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:15.090543 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bef4864-880d-4f1b-bce7-792deec6e8c7 service nova] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:15.090988 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bef4864-880d-4f1b-bce7-792deec6e8c7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:15.094378 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-5bef4864-880d-4f1b-bce7-792deec6e8c7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:15.120905 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bef4864-880d-4f1b-bce7-792deec6e8c7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:15.123137 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bef4864-880d-4f1b-bce7-792deec6e8c7 service nova] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:15.148903 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-564af720-daa7-46d1-a602-d9658c6c07e2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '32b1a8067ca14593a3490ab5ddf500f5'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:15.150254 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-564af720-daa7-46d1-a602-d9658c6c07e2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '32b1a8067ca14593a3490ab5ddf500f5'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:15.155639 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1431/2876] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:56:14 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:56:15.159952 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-564af720-daa7-46d1-a602-d9658c6c07e2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '32b1a8067ca14593a3490ab5ddf500f5'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:15.160936 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-564af720-daa7-46d1-a602-d9658c6c07e2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '32b1a8067ca14593a3490ab5ddf500f5'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:15.180162 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-564af720-daa7-46d1-a602-d9658c6c07e2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '32b1a8067ca14593a3490ab5ddf500f5'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:15.207902 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1446/2877] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 17:56:14 2022] DELETE /identity/v3/users/32b1a8067ca14593a3490ab5ddf500f5 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:56:15.508836 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bef4864-880d-4f1b-bce7-792deec6e8c7 service nova] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:15.510998 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-5bef4864-880d-4f1b-bce7-792deec6e8c7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:15.513569 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564af720-daa7-46d1-a602-d9658c6c07e2 admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:15.518165 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-564af720-daa7-46d1-a602-d9658c6c07e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:15.554617 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bef4864-880d-4f1b-bce7-792deec6e8c7 service nova] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:15.596999 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564af720-daa7-46d1-a602-d9658c6c07e2 admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:15.602366 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a643cfb6-2f2a-45f9-80d6-785873d1dfa8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43ad98a250a144099ccd18378aedd4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:15.602981 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a643cfb6-2f2a-45f9-80d6-785873d1dfa8 service neutron] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:15.603270 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a643cfb6-2f2a-45f9-80d6-785873d1dfa8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:15.603545 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a643cfb6-2f2a-45f9-80d6-785873d1dfa8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:15.605341 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-a643cfb6-2f2a-45f9-80d6-785873d1dfa8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:15.615762 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a643cfb6-2f2a-45f9-80d6-785873d1dfa8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:15.617261 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a643cfb6-2f2a-45f9-80d6-785873d1dfa8 service neutron] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:15.620472 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339de749-dcbd-4d4f-ae1c-94789c84c394 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:15.621190 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339de749-dcbd-4d4f-ae1c-94789c84c394 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:15.621499 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339de749-dcbd-4d4f-ae1c-94789c84c394 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:15.621782 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339de749-dcbd-4d4f-ae1c-94789c84c394 admin admin] PATH_INFO: `/v3/projects/4351320fd5184ac385fc403376bc9778` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:15.623282 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339de749-dcbd-4d4f-ae1c-94789c84c394 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4351320fd5184ac385fc403376bc9778)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:15.625258 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339de749-dcbd-4d4f-ae1c-94789c84c394 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:15.627943 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1447/2878] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:56:15 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:56:15.675065 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-339de749-dcbd-4d4f-ae1c-94789c84c394 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4351320fd5184ac385fc403376bc9778', 'request_id': 'req-339de749-dcbd-4d4f-ae1c-94789c84c394'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:15.680876 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-339de749-dcbd-4d4f-ae1c-94789c84c394 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4351320fd5184ac385fc403376bc9778', 'request_id': 'req-339de749-dcbd-4d4f-ae1c-94789c84c394'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:15.687249 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-339de749-dcbd-4d4f-ae1c-94789c84c394 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4351320fd5184ac385fc403376bc9778', 'request_id': 'req-339de749-dcbd-4d4f-ae1c-94789c84c394'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:15.699757 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-339de749-dcbd-4d4f-ae1c-94789c84c394 admin admin] The token cache is being invalidate because project 4351320fd5184ac385fc403376bc9778 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81759) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:56:15.700048 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-339de749-dcbd-4d4f-ae1c-94789c84c394 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:15.700794 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-339de749-dcbd-4d4f-ae1c-94789c84c394 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:15.702617 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1432/2879] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 17:56:15 2022] DELETE /identity/v3/projects/4351320fd5184ac385fc403376bc9778 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:56:15.706476 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a643cfb6-2f2a-45f9-80d6-785873d1dfa8 service neutron] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:15.707960 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-a643cfb6-2f2a-45f9-80d6-785873d1dfa8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:15.723660 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a643cfb6-2f2a-45f9-80d6-785873d1dfa8 service neutron] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:15.776152 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:56:15.791954 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b43e801-79be-4f85-87a3-20d6cfea6cf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:15.792635 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b43e801-79be-4f85-87a3-20d6cfea6cf3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:15.792920 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b43e801-79be-4f85-87a3-20d6cfea6cf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:15.793200 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b43e801-79be-4f85-87a3-20d6cfea6cf3 admin admin] PATH_INFO: `/v3/users/03766f29c1ef462f8d206c20ad26c8b5` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:15.794415 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b43e801-79be-4f85-87a3-20d6cfea6cf3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=03766f29c1ef462f8d206c20ad26c8b5)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:15.796356 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b43e801-79be-4f85-87a3-20d6cfea6cf3 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:15.845940 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-6b43e801-79be-4f85-87a3-20d6cfea6cf3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '03766f29c1ef462f8d206c20ad26c8b5'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:15.846598 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-6b43e801-79be-4f85-87a3-20d6cfea6cf3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '03766f29c1ef462f8d206c20ad26c8b5'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:15.853335 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-6b43e801-79be-4f85-87a3-20d6cfea6cf3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '03766f29c1ef462f8d206c20ad26c8b5'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:15.853967 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-6b43e801-79be-4f85-87a3-20d6cfea6cf3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '03766f29c1ef462f8d206c20ad26c8b5'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:15.866142 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-6b43e801-79be-4f85-87a3-20d6cfea6cf3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '03766f29c1ef462f8d206c20ad26c8b5'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:15.879198 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1448/2880] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 17:56:15 2022] DELETE /identity/v3/users/03766f29c1ef462f8d206c20ad26c8b5 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:56:16.168164 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339de749-dcbd-4d4f-ae1c-94789c84c394 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:16.172576 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-339de749-dcbd-4d4f-ae1c-94789c84c394 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:16.182939 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339de749-dcbd-4d4f-ae1c-94789c84c394 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:16.230932 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:56:16.252710 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2322be-fb68-42fa-924e-8cba7308f338 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:16.253356 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2322be-fb68-42fa-924e-8cba7308f338 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:16.253738 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2322be-fb68-42fa-924e-8cba7308f338 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:16.254015 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2322be-fb68-42fa-924e-8cba7308f338 admin admin] PATH_INFO: `/v3/projects/dd2a8882876b46ac85edeab6cd0b8805` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:16.255177 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2322be-fb68-42fa-924e-8cba7308f338 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dd2a8882876b46ac85edeab6cd0b8805)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:16.257866 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2322be-fb68-42fa-924e-8cba7308f338 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:16.309097 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-1f2322be-fb68-42fa-924e-8cba7308f338 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dd2a8882876b46ac85edeab6cd0b8805', 'request_id': 'req-1f2322be-fb68-42fa-924e-8cba7308f338'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:16.315207 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-1f2322be-fb68-42fa-924e-8cba7308f338 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dd2a8882876b46ac85edeab6cd0b8805', 'request_id': 'req-1f2322be-fb68-42fa-924e-8cba7308f338'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:16.322475 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-1f2322be-fb68-42fa-924e-8cba7308f338 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dd2a8882876b46ac85edeab6cd0b8805', 'request_id': 'req-1f2322be-fb68-42fa-924e-8cba7308f338'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:16.336572 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-1f2322be-fb68-42fa-924e-8cba7308f338 admin admin] The token cache is being invalidate because project dd2a8882876b46ac85edeab6cd0b8805 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81759) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:56:16.336881 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-1f2322be-fb68-42fa-924e-8cba7308f338 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:16.337484 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-1f2322be-fb68-42fa-924e-8cba7308f338 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:16.338994 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1433/2881] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 17:56:16 2022] DELETE /identity/v3/projects/dd2a8882876b46ac85edeab6cd0b8805 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:56:16.343833 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac58f35-3fb8-4496-9861-991a834a58b6 None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:16.344341 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac58f35-3fb8-4496-9861-991a834a58b6 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:16.344460 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac58f35-3fb8-4496-9861-991a834a58b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:16.363556 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-dac58f35-3fb8-4496-9861-991a834a58b6 None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:56:16.364783 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-dac58f35-3fb8-4496-9861-991a834a58b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:16.423626 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1449/2882] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:56:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:56:16.433418 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e269e7c-89f4-468e-9ff6-c02ee9918a2d None None] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:16.433698 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e269e7c-89f4-468e-9ff6-c02ee9918a2d None None] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:16.433991 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e269e7c-89f4-468e-9ff6-c02ee9918a2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:16.455952 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.auth.core [None req-8e269e7c-89f4-468e-9ff6-c02ee9918a2d None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81759) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:56:16.457121 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-8e269e7c-89f4-468e-9ff6-c02ee9918a2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:16.465975 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1434/2883] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:56:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:56:16.470196 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac58f35-3fb8-4496-9861-991a834a58b6 None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:16.480037 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac58f35-3fb8-4496-9861-991a834a58b6 None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:16.484672 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f64aaa-4e72-441d-8c55-a3835cf76bfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:16.485264 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f64aaa-4e72-441d-8c55-a3835cf76bfb admin admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:16.485542 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f64aaa-4e72-441d-8c55-a3835cf76bfb admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:16.485818 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f64aaa-4e72-441d-8c55-a3835cf76bfb admin admin] PATH_INFO: `/v3/domains` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:16.486463 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f64aaa-4e72-441d-8c55-a3835cf76bfb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81758) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:56:16.486763 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f64aaa-4e72-441d-8c55-a3835cf76bfb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:16.488352 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f64aaa-4e72-441d-8c55-a3835cf76bfb admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:16.500440 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1450/2884] 10.222.0.45 () {58 vars in 1153 bytes} [Wed Aug 24 17:56:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:56:16.504844 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e269e7c-89f4-468e-9ff6-c02ee9918a2d None None] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:16.514638 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e269e7c-89f4-468e-9ff6-c02ee9918a2d None None] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:16.519513 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e905f06-7486-4940-836e-89bea0b5eccc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:16.520125 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e905f06-7486-4940-836e-89bea0b5eccc admin admin] REQUEST_METHOD: `POST` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:16.520405 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e905f06-7486-4940-836e-89bea0b5eccc admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:16.520710 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e905f06-7486-4940-836e-89bea0b5eccc admin admin] PATH_INFO: `/v3/projects` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:16.521347 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e905f06-7486-4940-836e-89bea0b5eccc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:16.523781 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e905f06-7486-4940-836e-89bea0b5eccc admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:16.554895 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1435/2885] 10.222.0.45 () {60 vars in 1153 bytes} [Wed Aug 24 17:56:16 2022] POST /identity/v3/projects => generated 383 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:56:16.556122 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f64aaa-4e72-441d-8c55-a3835cf76bfb admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:16.565783 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f64aaa-4e72-441d-8c55-a3835cf76bfb admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:16.617585 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:56:16.625695 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0745d0-d92c-4646-b1a6-bb710871d284 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:16.626322 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0745d0-d92c-4646-b1a6-bb710871d284 admin admin] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:16.626972 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0745d0-d92c-4646-b1a6-bb710871d284 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:16.627411 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0745d0-d92c-4646-b1a6-bb710871d284 admin admin] PATH_INFO: `/v3/users` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:16.628035 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0745d0-d92c-4646-b1a6-bb710871d284 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:16.630352 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0745d0-d92c-4646-b1a6-bb710871d284 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:16.653276 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1451/2886] 10.222.0.45 () {60 vars in 1144 bytes} [Wed Aug 24 17:56:16 2022] POST /identity/v3/users => generated 292 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:56:16.657467 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e905f06-7486-4940-836e-89bea0b5eccc admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:16.668044 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e905f06-7486-4940-836e-89bea0b5eccc admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:16.672804 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36df0dd3-9609-435d-a27c-39429c1570ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:16.673421 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36df0dd3-9609-435d-a27c-39429c1570ea admin admin] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:16.673691 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36df0dd3-9609-435d-a27c-39429c1570ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:16.673965 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36df0dd3-9609-435d-a27c-39429c1570ea admin admin] PATH_INFO: `/v3/roles` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:16.674664 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36df0dd3-9609-435d-a27c-39429c1570ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:16.676265 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36df0dd3-9609-435d-a27c-39429c1570ea admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:16.687465 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1436/2887] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:56:16 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:56:16.691832 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0745d0-d92c-4646-b1a6-bb710871d284 admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:16.703565 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0745d0-d92c-4646-b1a6-bb710871d284 admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:16.712826 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcac3405-6829-40a5-bf47-ca2ceb7a6a2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:16.713830 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcac3405-6829-40a5-bf47-ca2ceb7a6a2a admin admin] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:16.714286 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcac3405-6829-40a5-bf47-ca2ceb7a6a2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:16.714731 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcac3405-6829-40a5-bf47-ca2ceb7a6a2a admin admin] PATH_INFO: `/v3/roles` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:16.715570 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcac3405-6829-40a5-bf47-ca2ceb7a6a2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:16.717454 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcac3405-6829-40a5-bf47-ca2ceb7a6a2a admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:16.731468 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1452/2888] 10.222.0.45 () {58 vars in 1122 bytes} [Wed Aug 24 17:56:16 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 17:56:16.735794 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36df0dd3-9609-435d-a27c-39429c1570ea admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:16.746432 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36df0dd3-9609-435d-a27c-39429c1570ea admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:16.753305 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa0280d-fcc5-471c-a14d-6c5aff595fb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:16.754280 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa0280d-fcc5-471c-a14d-6c5aff595fb9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:16.754567 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa0280d-fcc5-471c-a14d-6c5aff595fb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:16.754843 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa0280d-fcc5-471c-a14d-6c5aff595fb9 admin admin] PATH_INFO: `/v3/projects/0a389d974a9c465f8f5a8204b77be62b/users/fa2df13f963549078f92b91cf21da2f0/roles/02476a5b82eb4b188c18a5d706a3ef68` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:16.774770 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa0280d-fcc5-471c-a14d-6c5aff595fb9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a389d974a9c465f8f5a8204b77be62b, user_id=fa2df13f963549078f92b91cf21da2f0, role_id=02476a5b82eb4b188c18a5d706a3ef68)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:16.777985 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa0280d-fcc5-471c-a14d-6c5aff595fb9 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:16.795781 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1437/2889] 10.222.0.45 () {60 vars in 1483 bytes} [Wed Aug 24 17:56:16 2022] PUT /identity/v3/projects/0a389d974a9c465f8f5a8204b77be62b/users/fa2df13f963549078f92b91cf21da2f0/roles/02476a5b82eb4b188c18a5d706a3ef68 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:56:16.801005 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f4b81a-5474-40bd-a15b-a3c6d713adba None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:16.801465 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f4b81a-5474-40bd-a15b-a3c6d713adba None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:16.801905 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f4b81a-5474-40bd-a15b-a3c6d713adba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:16.821997 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-12f4b81a-5474-40bd-a15b-a3c6d713adba None None] MFA Rules not processed for user `cc2669e1aa494dbcb7164d2e5e46378c`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:56:16.823193 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-12f4b81a-5474-40bd-a15b-a3c6d713adba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:16.907113 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1453/2890] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:56:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:56:16.917431 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa0280d-fcc5-471c-a14d-6c5aff595fb9 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:16.918845 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-3aa0280d-fcc5-471c-a14d-6c5aff595fb9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:16.990940 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa0280d-fcc5-471c-a14d-6c5aff595fb9 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:16.994754 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703dc785-a888-424d-83d5-4619cad1b5b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43ad98a250a144099ccd18378aedd4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:16.995443 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703dc785-a888-424d-83d5-4619cad1b5b2 service neutron] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:16.995792 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703dc785-a888-424d-83d5-4619cad1b5b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:16.996254 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703dc785-a888-424d-83d5-4619cad1b5b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:17.007594 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703dc785-a888-424d-83d5-4619cad1b5b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:17.009356 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703dc785-a888-424d-83d5-4619cad1b5b2 service neutron] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:17.019233 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1438/2891] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:56:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:56:17.656979 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89b0ca9-ed52-403a-ac40-fb4ccd4b5c0c None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:17.657804 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89b0ca9-ed52-403a-ac40-fb4ccd4b5c0c None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:17.658656 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89b0ca9-ed52-403a-ac40-fb4ccd4b5c0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:17.713912 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-e89b0ca9-ed52-403a-ac40-fb4ccd4b5c0c None None] MFA Rules not processed for user `fa2df13f963549078f92b91cf21da2f0`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:56:17.715575 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-e89b0ca9-ed52-403a-ac40-fb4ccd4b5c0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:17.812707 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1454/2892] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:56:17 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:56:17.822466 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703dc785-a888-424d-83d5-4619cad1b5b2 service neutron] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:17.824749 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-703dc785-a888-424d-83d5-4619cad1b5b2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:17.841827 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:56:17.903623 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703dc785-a888-424d-83d5-4619cad1b5b2 service neutron] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:17.907277 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cd6696-ced4-4f25-a228-9130e31611cb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59be342541264942ad87f60ff4de6c8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:17.907892 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cd6696-ced4-4f25-a228-9130e31611cb service cinder] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:17.908176 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cd6696-ced4-4f25-a228-9130e31611cb service cinder] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:17.908450 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cd6696-ced4-4f25-a228-9130e31611cb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:17.918481 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5cd6696-ced4-4f25-a228-9130e31611cb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:17.919944 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5cd6696-ced4-4f25-a228-9130e31611cb service cinder] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:17.930066 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1439/2893] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:56:17 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:56:19.343670 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2009c7cb-fe68-4c0e-ae6e-0aef8f41d1e5 None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:19.344145 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2009c7cb-fe68-4c0e-ae6e-0aef8f41d1e5 None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:19.344264 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2009c7cb-fe68-4c0e-ae6e-0aef8f41d1e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:19.363181 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-2009c7cb-fe68-4c0e-ae6e-0aef8f41d1e5 None None] MFA Rules not processed for user `fa2df13f963549078f92b91cf21da2f0`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:56:19.364378 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-2009c7cb-fe68-4c0e-ae6e-0aef8f41d1e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:19.371272 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1455/2894] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:56:19 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:56:19.478077 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cd6696-ced4-4f25-a228-9130e31611cb service cinder] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:19.487516 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cd6696-ced4-4f25-a228-9130e31611cb service cinder] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:19.491092 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d3dff7-273c-4639-9a08-c5ac11a0834b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43ad98a250a144099ccd18378aedd4b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:19.491680 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d3dff7-273c-4639-9a08-c5ac11a0834b service neutron] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:19.491955 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d3dff7-273c-4639-9a08-c5ac11a0834b service neutron] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:19.492224 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d3dff7-273c-4639-9a08-c5ac11a0834b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:19.493958 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-23d3dff7-273c-4639-9a08-c5ac11a0834b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:19.566501 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d3dff7-273c-4639-9a08-c5ac11a0834b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:19.567949 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d3dff7-273c-4639-9a08-c5ac11a0834b service neutron] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:19.579860 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1440/2895] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:56:19 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:56:23.476353 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6990a180-1ff3-4657-b7a7-4e590a8686ed None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:23.476917 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6990a180-1ff3-4657-b7a7-4e590a8686ed None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:23.476917 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6990a180-1ff3-4657-b7a7-4e590a8686ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:23.498642 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-6990a180-1ff3-4657-b7a7-4e590a8686ed None None] MFA Rules not processed for user `ea143d042ee94ae7a4544d90d18db7db`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:56:23.499722 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-6990a180-1ff3-4657-b7a7-4e590a8686ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:23.571466 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1456/2896] 10.222.0.45 () {58 vars in 967 bytes} [Wed Aug 24 17:56:23 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:56:23.619433 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d3dff7-273c-4639-9a08-c5ac11a0834b service neutron] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:23.620915 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-23d3dff7-273c-4639-9a08-c5ac11a0834b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:23.693336 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d3dff7-273c-4639-9a08-c5ac11a0834b service neutron] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:23.698376 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249d042a-096c-416f-8b3f-7d3519f6a835 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90f0772b64c84faaa689e7272fa97898', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:23.698990 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249d042a-096c-416f-8b3f-7d3519f6a835 service swift] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:23.699271 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249d042a-096c-416f-8b3f-7d3519f6a835 service swift] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:23.699548 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249d042a-096c-416f-8b3f-7d3519f6a835 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:23.707807 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249d042a-096c-416f-8b3f-7d3519f6a835 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:23.710031 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249d042a-096c-416f-8b3f-7d3519f6a835 service swift] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:23.718043 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1441/2897] 10.222.0.45 () {60 vars in 1384 bytes} [Wed Aug 24 17:56:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 24 17:56:26.989525 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6990a180-1ff3-4657-b7a7-4e590a8686ed None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:26.991080 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-6990a180-1ff3-4657-b7a7-4e590a8686ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:26.996181 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:56:27.111086 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6990a180-1ff3-4657-b7a7-4e590a8686ed None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:27.120181 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5433823d-2421-41da-91e8-14c759160b9b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37def4721b554b9ebb7704a19642d15b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:27.121265 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5433823d-2421-41da-91e8-14c759160b9b service placement] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:27.121548 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5433823d-2421-41da-91e8-14c759160b9b service placement] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:27.121821 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5433823d-2421-41da-91e8-14c759160b9b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:27.123842 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-5433823d-2421-41da-91e8-14c759160b9b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:27.143333 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5433823d-2421-41da-91e8-14c759160b9b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:27.145777 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5433823d-2421-41da-91e8-14c759160b9b service placement] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:27.158894 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1457/2898] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:56:26 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:56:30.367617 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249d042a-096c-416f-8b3f-7d3519f6a835 service swift] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:30.380196 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249d042a-096c-416f-8b3f-7d3519f6a835 service swift] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:30.386865 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f575efba-ada8-4196-b327-a6b6368d5227 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37def4721b554b9ebb7704a19642d15b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:30.387697 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f575efba-ada8-4196-b327-a6b6368d5227 service placement] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:30.388317 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f575efba-ada8-4196-b327-a6b6368d5227 service placement] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:30.388909 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f575efba-ada8-4196-b327-a6b6368d5227 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:30.391225 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-f575efba-ada8-4196-b327-a6b6368d5227 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:30.407078 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f575efba-ada8-4196-b327-a6b6368d5227 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:30.409266 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f575efba-ada8-4196-b327-a6b6368d5227 service placement] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:30.422882 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1442/2899] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:56:30 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:56:30.426500 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5433823d-2421-41da-91e8-14c759160b9b service placement] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:30.428074 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-5433823d-2421-41da-91e8-14c759160b9b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:30.451514 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5433823d-2421-41da-91e8-14c759160b9b service placement] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:30.456670 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fe90db-f6f7-4f2a-baa1-34ffe43d118c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:30.457321 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fe90db-f6f7-4f2a-baa1-34ffe43d118c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:30.457671 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fe90db-f6f7-4f2a-baa1-34ffe43d118c admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:30.457899 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fe90db-f6f7-4f2a-baa1-34ffe43d118c admin admin] PATH_INFO: `/v3/users/127dbee9996d4c9e8f8603edaa579e36` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:30.459575 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fe90db-f6f7-4f2a-baa1-34ffe43d118c admin admin] RBAC: Authorizing `identity:delete_user(user_id=127dbee9996d4c9e8f8603edaa579e36)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:30.461534 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fe90db-f6f7-4f2a-baa1-34ffe43d118c admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:30.507491 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-b2fe90db-f6f7-4f2a-baa1-34ffe43d118c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '127dbee9996d4c9e8f8603edaa579e36'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:30.508157 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-b2fe90db-f6f7-4f2a-baa1-34ffe43d118c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '127dbee9996d4c9e8f8603edaa579e36'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:30.515689 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-b2fe90db-f6f7-4f2a-baa1-34ffe43d118c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '127dbee9996d4c9e8f8603edaa579e36'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:30.516230 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-b2fe90db-f6f7-4f2a-baa1-34ffe43d118c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '127dbee9996d4c9e8f8603edaa579e36'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:30.530943 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-b2fe90db-f6f7-4f2a-baa1-34ffe43d118c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '127dbee9996d4c9e8f8603edaa579e36'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:30.544882 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1458/2900] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 17:56:30 2022] DELETE /identity/v3/users/127dbee9996d4c9e8f8603edaa579e36 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:56:30.722692 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f575efba-ada8-4196-b327-a6b6368d5227 service placement] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:30.740337 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f575efba-ada8-4196-b327-a6b6368d5227 service placement] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:30.750150 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=100 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:56:30.801846 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e83c175-782f-489e-b208-bbfecb4e3180 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:30.802438 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e83c175-782f-489e-b208-bbfecb4e3180 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:30.802717 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e83c175-782f-489e-b208-bbfecb4e3180 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:30.802985 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e83c175-782f-489e-b208-bbfecb4e3180 admin admin] PATH_INFO: `/v3/projects/2de47efc59f04643ab02651d40b77067` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:30.804214 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e83c175-782f-489e-b208-bbfecb4e3180 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2de47efc59f04643ab02651d40b77067)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:30.806147 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e83c175-782f-489e-b208-bbfecb4e3180 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:30.853699 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-8e83c175-782f-489e-b208-bbfecb4e3180 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2de47efc59f04643ab02651d40b77067', 'request_id': 'req-8e83c175-782f-489e-b208-bbfecb4e3180'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:30.859045 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-8e83c175-782f-489e-b208-bbfecb4e3180 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2de47efc59f04643ab02651d40b77067', 'request_id': 'req-8e83c175-782f-489e-b208-bbfecb4e3180'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:30.865094 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-8e83c175-782f-489e-b208-bbfecb4e3180 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2de47efc59f04643ab02651d40b77067', 'request_id': 'req-8e83c175-782f-489e-b208-bbfecb4e3180'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:30.877266 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-8e83c175-782f-489e-b208-bbfecb4e3180 admin admin] The token cache is being invalidate because project 2de47efc59f04643ab02651d40b77067 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81759) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:56:30.877549 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-8e83c175-782f-489e-b208-bbfecb4e3180 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:30.878157 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-8e83c175-782f-489e-b208-bbfecb4e3180 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:30.879657 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1443/2901] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 17:56:30 2022] DELETE /identity/v3/projects/2de47efc59f04643ab02651d40b77067 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:56:30.884437 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fe90db-f6f7-4f2a-baa1-34ffe43d118c admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:30.885888 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-b2fe90db-f6f7-4f2a-baa1-34ffe43d118c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:30.899519 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fe90db-f6f7-4f2a-baa1-34ffe43d118c admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:30.954944 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd958024-7c77-4018-92e8-4bf2e402db25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:30.955570 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd958024-7c77-4018-92e8-4bf2e402db25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:30.955851 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd958024-7c77-4018-92e8-4bf2e402db25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:30.956127 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd958024-7c77-4018-92e8-4bf2e402db25 admin admin] PATH_INFO: `/v3/users/e5da8016c3624ee5aa9766f0dbdca61e` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:30.957276 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd958024-7c77-4018-92e8-4bf2e402db25 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e5da8016c3624ee5aa9766f0dbdca61e)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:30.959740 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd958024-7c77-4018-92e8-4bf2e402db25 admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:30.964384 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:56:31.010797 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-fd958024-7c77-4018-92e8-4bf2e402db25 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e5da8016c3624ee5aa9766f0dbdca61e'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:31.011453 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-fd958024-7c77-4018-92e8-4bf2e402db25 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e5da8016c3624ee5aa9766f0dbdca61e'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:31.018332 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-fd958024-7c77-4018-92e8-4bf2e402db25 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e5da8016c3624ee5aa9766f0dbdca61e'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:31.019002 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-fd958024-7c77-4018-92e8-4bf2e402db25 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e5da8016c3624ee5aa9766f0dbdca61e'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:31.031107 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-fd958024-7c77-4018-92e8-4bf2e402db25 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e5da8016c3624ee5aa9766f0dbdca61e'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:31.043397 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1459/2902] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 17:56:30 2022] DELETE /identity/v3/users/e5da8016c3624ee5aa9766f0dbdca61e => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:56:31.214328 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e83c175-782f-489e-b208-bbfecb4e3180 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:31.216089 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-8e83c175-782f-489e-b208-bbfecb4e3180 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:31.236994 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e83c175-782f-489e-b208-bbfecb4e3180 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:31.312407 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3592eae5-ce5c-4217-b65a-253ff61db5cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:31.313108 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3592eae5-ce5c-4217-b65a-253ff61db5cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:31.313358 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3592eae5-ce5c-4217-b65a-253ff61db5cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:31.316799 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3592eae5-ce5c-4217-b65a-253ff61db5cb admin admin] PATH_INFO: `/v3/projects/f0b13a0288ad488582b72921a21c51a1` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:31.319026 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3592eae5-ce5c-4217-b65a-253ff61db5cb admin admin] RBAC: Authorizing `identity:delete_project(project_id=f0b13a0288ad488582b72921a21c51a1)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:31.321277 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3592eae5-ce5c-4217-b65a-253ff61db5cb admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:31.369646 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:56:31.372670 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-3592eae5-ce5c-4217-b65a-253ff61db5cb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f0b13a0288ad488582b72921a21c51a1', 'request_id': 'req-3592eae5-ce5c-4217-b65a-253ff61db5cb'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:31.379722 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-3592eae5-ce5c-4217-b65a-253ff61db5cb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f0b13a0288ad488582b72921a21c51a1', 'request_id': 'req-3592eae5-ce5c-4217-b65a-253ff61db5cb'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:31.387296 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-3592eae5-ce5c-4217-b65a-253ff61db5cb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f0b13a0288ad488582b72921a21c51a1', 'request_id': 'req-3592eae5-ce5c-4217-b65a-253ff61db5cb'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:31.399671 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-3592eae5-ce5c-4217-b65a-253ff61db5cb admin admin] The token cache is being invalidate because project f0b13a0288ad488582b72921a21c51a1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81759) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:56:31.399966 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-3592eae5-ce5c-4217-b65a-253ff61db5cb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:31.400610 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-3592eae5-ce5c-4217-b65a-253ff61db5cb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:31.402081 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1444/2903] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 17:56:31 2022] DELETE /identity/v3/projects/f0b13a0288ad488582b72921a21c51a1 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:56:41.040041 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=24,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:56:41.396737 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:DELETE=4,keystone:SELECT=7,keystone:INSERT=1 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:57:34.606986 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c648b4-3275-47a4-ada1-a84f2415c59d None None] REQUEST_METHOD: `POST` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:57:34.607513 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c648b4-3275-47a4-ada1-a84f2415c59d None None] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:57:34.607513 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c648b4-3275-47a4-ada1-a84f2415c59d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:57:34.626435 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.auth.core [None req-14c648b4-3275-47a4-ada1-a84f2415c59d None None] MFA Rules not processed for user `fa2df13f963549078f92b91cf21da2f0`. Rule list: `[]` (Enabled: `True`). {{(pid=81758) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:57:34.627626 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-14c648b4-3275-47a4-ada1-a84f2415c59d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:57:34.679331 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1460/2904] 10.222.0.45 () {58 vars in 958 bytes} [Wed Aug 24 17:57:34 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:57:44.672420 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=25 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:58:40.240736 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3592eae5-ce5c-4217-b65a-253ff61db5cb admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:58:40.242171 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-3592eae5-ce5c-4217-b65a-253ff61db5cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:58:40.342424 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3592eae5-ce5c-4217-b65a-253ff61db5cb admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:58:40.347203 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8e982b-dd8c-4417-a44f-ad926d0573e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:58:40.347849 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8e982b-dd8c-4417-a44f-ad926d0573e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:58:40.348129 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8e982b-dd8c-4417-a44f-ad926d0573e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:58:40.348402 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8e982b-dd8c-4417-a44f-ad926d0573e4 admin admin] PATH_INFO: `/v3/users/fa2df13f963549078f92b91cf21da2f0` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:58:40.350109 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8e982b-dd8c-4417-a44f-ad926d0573e4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fa2df13f963549078f92b91cf21da2f0)` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:58:40.352037 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8e982b-dd8c-4417-a44f-ad926d0573e4 admin admin] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:58:40.522031 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-2e8e982b-dd8c-4417-a44f-ad926d0573e4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fa2df13f963549078f92b91cf21da2f0'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:58:40.522609 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-2e8e982b-dd8c-4417-a44f-ad926d0573e4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fa2df13f963549078f92b91cf21da2f0'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:58:40.529456 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-2e8e982b-dd8c-4417-a44f-ad926d0573e4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fa2df13f963549078f92b91cf21da2f0'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:58:40.530086 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-2e8e982b-dd8c-4417-a44f-ad926d0573e4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fa2df13f963549078f92b91cf21da2f0'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:58:40.537189 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.notifications [None req-2e8e982b-dd8c-4417-a44f-ad926d0573e4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fa2df13f963549078f92b91cf21da2f0'} {{(pid=81759) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:58:40.554216 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1445/2905] 10.222.0.45 () {58 vars in 1224 bytes} [Wed Aug 24 17:58:40 2022] DELETE /identity/v3/users/fa2df13f963549078f92b91cf21da2f0 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:58:40.722480 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c648b4-3275-47a4-ada1-a84f2415c59d None None] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:58:40.758417 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c648b4-3275-47a4-ada1-a84f2415c59d None None] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:58:40.814454 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c316dc3-3fb6-4c33-bc2f-80dcf14e36ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc2669e1aa494dbcb7164d2e5e46378c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e847003e2ef47df81d36ac4caad47f0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:58:40.815227 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c316dc3-3fb6-4c33-bc2f-80dcf14e36ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:58:40.816055 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c316dc3-3fb6-4c33-bc2f-80dcf14e36ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:58:40.816055 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c316dc3-3fb6-4c33-bc2f-80dcf14e36ff admin admin] PATH_INFO: `/v3/projects/0a389d974a9c465f8f5a8204b77be62b` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:58:40.817129 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c316dc3-3fb6-4c33-bc2f-80dcf14e36ff admin admin] RBAC: Authorizing `identity:delete_project(project_id=0a389d974a9c465f8f5a8204b77be62b)` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:58:40.819737 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c316dc3-3fb6-4c33-bc2f-80dcf14e36ff admin admin] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:58:40.867477 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-1c316dc3-3fb6-4c33-bc2f-80dcf14e36ff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0a389d974a9c465f8f5a8204b77be62b', 'request_id': 'req-1c316dc3-3fb6-4c33-bc2f-80dcf14e36ff'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:58:40.874024 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-1c316dc3-3fb6-4c33-bc2f-80dcf14e36ff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0a389d974a9c465f8f5a8204b77be62b', 'request_id': 'req-1c316dc3-3fb6-4c33-bc2f-80dcf14e36ff'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:58:40.880408 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-1c316dc3-3fb6-4c33-bc2f-80dcf14e36ff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0a389d974a9c465f8f5a8204b77be62b', 'request_id': 'req-1c316dc3-3fb6-4c33-bc2f-80dcf14e36ff'} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:58:40.890511 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-1c316dc3-3fb6-4c33-bc2f-80dcf14e36ff admin admin] The token cache is being invalidate because project 0a389d974a9c465f8f5a8204b77be62b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81758) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:58:40.890796 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-1c316dc3-3fb6-4c33-bc2f-80dcf14e36ff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:58:40.891412 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.notifications [None req-1c316dc3-3fb6-4c33-bc2f-80dcf14e36ff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81758) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:58:40.892934 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1461/2906] 10.222.0.45 () {58 vars in 1233 bytes} [Wed Aug 24 17:58:40 2022] DELETE /identity/v3/projects/0a389d974a9c465f8f5a8204b77be62b => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:58:41.851786 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8e982b-dd8c-4417-a44f-ad926d0573e4 admin admin] Authenticating user token {{(pid=81759) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:58:41.853366 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-2e8e982b-dd8c-4417-a44f-ad926d0573e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:58:41.904798 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:58:41.961207 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8e982b-dd8c-4417-a44f-ad926d0573e4 admin admin] Validating token access rules against request {{(pid=81759) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:58:42.020587 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b1dc86-7d27-4013-bf5c-55db7d7626fa service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37def4721b554b9ebb7704a19642d15b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81759) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:58:42.021185 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b1dc86-7d27-4013-bf5c-55db7d7626fa service placement] REQUEST_METHOD: `GET` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:58:42.021459 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b1dc86-7d27-4013-bf5c-55db7d7626fa service placement] SCRIPT_NAME: `/identity` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:58:42.021726 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b1dc86-7d27-4013-bf5c-55db7d7626fa service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81759) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:58:42.023427 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.fernet_utils [None req-51b1dc86-7d27-4013-bf5c-55db7d7626fa service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81759) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:58:42.034288 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b1dc86-7d27-4013-bf5c-55db7d7626fa service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:58:42.035847 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b1dc86-7d27-4013-bf5c-55db7d7626fa service placement] RBAC: Authorization granted {{(pid=81759) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:58:42.046012 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: [pid: 81759|app: 0|req: 1446/2907] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 17:58:41 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:58:50.887582 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=59,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:58:52.041908 dv-cinder-835384-24-iscsi devstack@keystone.service[81759]: DEBUG dbcounter [-] [81759] Writing DB stats keystone:SELECT=53 {{(pid=81759) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 18:03:46.866714 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c316dc3-3fb6-4c33-bc2f-80dcf14e36ff admin admin] Authenticating user token {{(pid=81758) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 18:03:46.868139 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.fernet_utils [None req-1c316dc3-3fb6-4c33-bc2f-80dcf14e36ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81758) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 18:03:46.883525 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c316dc3-3fb6-4c33-bc2f-80dcf14e36ff admin admin] Validating token access rules against request {{(pid=81758) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 18:03:46.888518 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f7a294-a476-4b47-981d-2db1180a2aa7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37def4721b554b9ebb7704a19642d15b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ad8a32bd9d2434097bd08b111914ca1', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81758) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 18:03:46.889146 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f7a294-a476-4b47-981d-2db1180a2aa7 service placement] REQUEST_METHOD: `GET` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 18:03:46.889427 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f7a294-a476-4b47-981d-2db1180a2aa7 service placement] SCRIPT_NAME: `/identity` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 18:03:46.889701 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f7a294-a476-4b47-981d-2db1180a2aa7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81758) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 18:03:46.898328 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f7a294-a476-4b47-981d-2db1180a2aa7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 18:03:46.899911 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f7a294-a476-4b47-981d-2db1180a2aa7 service placement] RBAC: Authorization granted {{(pid=81758) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 18:03:46.909787 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: [pid: 81758|app: 0|req: 1462/2908] 10.222.0.45 () {60 vars in 1365 bytes} [Wed Aug 24 18:03:46 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 18:03:56.905544 dv-cinder-835384-24-iscsi devstack@keystone.service[81758]: DEBUG dbcounter [-] [81758] Writing DB stats keystone:SELECT=6 {{(pid=81758) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}